Let's say we have a linear model:

y = beta + pc1 + x3 + x4

'pc1' is a principal component of x1 and x2 which have a high positive correlation(in the training data).

Would this model still be valid if the test data has a high but negative correlation between x1 and x2?

  • 1
    $\begingroup$ high positive/negative correlation Yes, if we admit that you somehow mixed up with the sign of one of the variables in the test dataset. Frequent situation in psychology, by the way. $\endgroup$
    – ttnphns
    Commented Mar 1, 2018 at 16:42

1 Answer 1


This doesn't seem to be a dimensionality-reduction problem. The problem that I see is with your sampling method.

To build your training set, as you said, you have sampled your data in such a way that $x_1$ and $x_2$ have different (opposite) correlations, compared to the test set. This is not a reliable (good representative) training set and your learning model won't be able to perform well on the test set, no matter how well your PCA managed to find the principal components.

My guess for how to correct this is that perhaps you made a minor programming mistake when you divided your data into test and training, since $x_1$ and $x_2$ are still highly correlated, but oppositely. You can make sure by comparing their magnitude of correlation. If they are equal, then try to find the minor mistake.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.