R train random forest for positive or negative predicitve value, not accuracy I am working with random forests on financial data (predicting if stock rises versus falls).
I figured out that I get better performance, if I build one model for "rising" and one for "falling".
When building the models with best features sets I then just manipulate the cut-off values of my probabilities (e.g. > 0.85 for "rising").
Yet, is there a way to a priori train random forests (or other models) for positive predictive value performance (so. it should only predict "rise" with a low false positive rate - I dont care about accuracy)
 A: In a portfolio optimizing strategy, it is an advantage to predict both the magnitude and probability of rise and falls.
Regression solution: Convert your time-series prices into a stationary measure such as log difference or relative change. The RF predicted change of prices will both incorporate magnitude and pseudo-probability and therefore a reasonable basis for ranking stocks.
Binary solution: You could convert your time-series into rise and fall events and classify these. Train e.g. 500 trees and use the vote ratio's to rank your most and least promising stocks. In a realistic situation your best vote ratio would be something like 270-230 as it is really difficult to predict well. You do not have to retrain to manipulate cut-offs. Just extract vote-ratio and implement your own rule (cut-off, ranking, etc.)
You can plot ROC on your predictions vs. outcome to learn what a good cutoff would be. Stock price predictions is imperfect modelling. You will be happy to predict a small component of the total volatility. You will not find a subset of predictions with "low false negative". Accept that you only can know very little about many events. Active trading on predictions should be hedged over thousands of investments, to ensure a high probability of a positive net income.
You need to give feature engineering some attention. Only inputting prior days directly will give you a mediocre model. You need to build a outstanding model to consistently outperform the market. If you just could take a of-the-shelf model and set it to make safe predictions, someone would have done that before you and it would already be incorporated in the market pricing.
