First steps learning to predict financial timeseries using machine learning

Question

I am trying to get a grasp on how to use machine learning to predict financial timeseries 1 or more steps into the future.

I have a financial timeseries with some descriptive data and I would like to form a model and then use the model to predict n-steps ahead.

What I have been doing so far is:

getSymbols("GOOG")

GOOG$sma <- SMA(Cl(GOOG))
GOOG$range <- GOOG$GOOG.High-GOOG$GOOG.Low

tail(GOOG)

           GOOG.Open GOOG.High GOOG.Low GOOG.Close GOOG.Volume GOOG.Adjusted     sma range
2013-05-07    863.01    863.87   850.67     857.23     1959000        857.23 828.214 13.20
2013-05-08    857.00    873.88   852.91     873.63     2468300        873.63 834.232 20.97
2013-05-09    870.84    879.66   868.23     871.48     2200600        871.48 840.470 11.43
2013-05-10    875.31    880.54   872.16     880.23     1897700        880.23 848.351  8.38
2013-05-13    878.89    882.47   873.38     877.53     1448500        877.53 854.198  9.09
2013-05-14    877.50    888.69   877.14     887.10     1579300        887.10 860.451 11.55

Then I have fitted a randomForest model to this data.

fit <- randomForest(GOOG$GOOG.Close ~ GOOG$sma + GOOG$range, GOOG)

Which seems to fit surprisingly well:

> fit

Call:
 randomForest(formula = GOOG$GOOG.Close ~ GOOG$sma + GOOG$range,      data = GOOG) 
               Type of random forest: regression
                     Number of trees: 500
No. of variables tried at each split: 1

          Mean of squared residuals: 353.9844
                    % Var explained: 97.28

And tried to use it to predict:

predict(fit, GOOG, n.ahead=2)

But this prediction ofc did not work.

I try to predict the Close, should I lag the other variables by as many steps as I want the prediction, before fitting the model?

Probably a lot of other stuff I should take into account as well but these are really my first steps trying out machine-learning.

Answer 1

My first observation is that you did not lag the inputs relative to the closing price and that is why you observed such good fit. The SMA (simple moving average) uses the closing price in its calculation and the high low range encompasses the closing price, so using them to predict the closing price imparts a look ahead bias. My opinion is that if you are trying to predict the closing price two days ahead you should build your model with inputs that are lagged from the closing price by at least two days. Some of the inputs may be lagged by more than two days, but I would start simple and try and use just a handful of inputs.

As far as your objective to predict closing price, I think that closing prices are too noisy to be used as target variables and using them will lead to overfitting or optimization of the wrong objective. Instead I would start by smoothing the closing price with a moving average and then predicting the direction of price change over the next two days. For example i might replace the close with a 5 day SMA of the close and then code the price change of the SMA as 1 if it was positive over the next two days and 0 otherwise. Because the output variable is now coded as a 1 or a 0 this is a good problem to try and solve with the random forest function you were using. You could also try some other classification algoritms like logistic regression, neural networks, and SVMs and maybe combine a few into an ensemble to improve your performance. This is still a difficult problem to solve without overfitting, but it is a step in the right direction. Another word of caution is that your final model could have amazing accuracy at classifying the next two days as either positive or negative, but still lose money because it classified a few large moves incorrectly.

I would also recommend building your model on more than one security so that the machine learning algorithm does not hone in on the idiosyncrasies of one stock. I would start with at least 5 stocks that are not highly correlated to eachother.

Trading on the Edge by Guido Deboeck is a good place to start for exploring the applications of machine learning to financial time series prediction. It's an older book so it is way behind the technology we have available today but it is a good start. I would also recommend New Trading Systems and Methods by Kaufman and Expert Trading Systems by John Wolberg.