I'm going to make an assumption here: When you say "the test data is significantly different from the training data" I am taking you to mean that for some reason you ran a goodness of fit test (e.g., Kolmogorov Two-sample test) and rejected the null. If that isn't what you mean, what do you mean? If that is what you mean, what test did you run (and why)?
So, given the assumption, the first question I would ask is, "Could this be a case of rejecting a true null?" To answer that question, I need to know how the training and test data came to be. If the training data are a random sample of the full data, then very clearly this IS a case of rejecting a true null.
If they came into existence via different processes, then you've established that the training data and test data are different. To me, that means that your classification (prediction) model is probably biased. Think about it in terms of a simple linear regression model. If you got your training data on the relationship to predict H.S. GPA fom IQ at a Mensa chapter, would you apply the model to the kids at Central High School? CHS students almost surely have a broader range of IQ scores than the Mensa chapter. The Mensa chapter almost certainly has a broader range of ages present than does CHS. You have to do serious extrapolation to apply the model estimates to CHS.
You have to take it on faith that the extrapolation is valid, that is, that the model you fit continues to hold in the new circumstances. So, my first issue is model bias (lack of fit bias of a sort) and that by itself would make me avoid applying the model to the test data.