I have a (relatively) large feature space - about 200 variables, and +1 million observations. My dependent variable is forward price return (raw, not log-transformed).
I apply a PCA transform of the independent variables, and subsequently plot observations against the first two PC dimensions, with blue for positive forward price move, red for negative forward price move, and dot size scaled to magnitude of the forward price move
For medium-magnitude returns (those within 2-sdev of mean), there does seem to be a nice separation in the majority of the positive/negative cases (axes on all plots are identical).
The same separation is true for large-magnitude returns - however, the regions are essentially flipped relative to the smaller/medium-magnitude moves.
When I move on to regression, logistic regression works reasonably well in a two-case model, but doesn't seem to be able to draw any further distinction when I create multi-class models (large-up;large-down;reg-up,reg-down), or when I try to predict magnitude only (irrespective of direction).
Any thoughts/suggestions on how I might proceed?