I have a (relatively) large feature space - about 200 variables, and +1 million observations. My dependent variable is forward price return (raw, not log-transformed).
I apply a PCA transform of the independent variables, and subsequently plot observations against the first two PC dimensions, with blue for positive forward price move, red for negative forward price move, and dot size scaled to magnitude of the forward price move

medium-magnitude forward returns

For medium-magnitude returns (those within 2-sdev of mean), there does seem to be a nice separation in the majority of the positive/negative cases (axes on all plots are identical).
The same separation is true for large-magnitude returns - however, the regions are essentially flipped relative to the smaller/medium-magnitude moves.

large-magnitude forward price moves

When I move on to regression, logistic regression works reasonably well in a two-case model, but doesn't seem to be able to draw any further distinction when I create multi-class models (large-up;large-down;reg-up,reg-down), or when I try to predict magnitude only (irrespective of direction).

Any thoughts/suggestions on how I might proceed?

  • $\begingroup$ Scale things. Most probably in your multi-class model the variance from the "large-magnitude returns" trumps any discriminatory properties found in the "smaller/medium-magnitude" moves. Yes, PCA does not make any strong Gaussian assumptions but data with extreme values (as returns/prices/etc.) can render its results suboptimal. $\endgroup$ – usεr11852 Apr 25 '17 at 21:03
  • $\begingroup$ @usεr11852 - thanks. The features (independent variables) are scaled, continuous -1:1. The dependent variable (y) is not included anywhere in the PCA transform. My observation/question is that when the feature space is transformed, the separation between y values (pos v neg forward price moves) is relatively well defined. However, as moves get larger, the pos/neg regions effectively reverse themselves.. $\endgroup$ – BrauHaus May 3 '17 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.