In the Figure1 below, the Training data is "red" and and Test data is "black". Is it correct to assume that any binary classifier will not generlize well? (keep in mind that the colors in the figure are not class labels..)
My rationale is that the both datasets have a different underlying mathematical structure (as visualized by tSNE), and thus a classifier trained in either one of these datasets will not perform well if tested on the other.
EDIT1: As a followup analysis (cannot post image due to low reputation), I individually filtered out all variables whose distribution is different in both datasets. The resulting figure appears much more "well mixed" (i.e. a good mix of red and black points). This "reduced" dataset is thus more suitable for classification, correct? Nonetheless I wonder if all classifiers can implicitly handle such covariate shifts?