0
$\begingroup$

I am quite new to the theory of RNNs so please excuse me if the question is trivial. I am trying to fit a multivariate LSTM neural network to predict stock prices from a firm in the S&P 500 list. I have seen many applications where the opening price is forecasted using the stock's own closing price, trading volume, highest and lowest trading prices.

However, I was wondering whether it would be sensible to try and forecast the firm's opening price using other firms' opening prices (specifically, I was thinking of other firms in the same sector in the S&P list, e.g. healthcare or IT sectors depending on the chosen firm). Since prices in the same sector are likely to be highly correlated I thought this could be a good approach. However, this would create a dataset with many features (much more than in the examples I have seen so far) and I am thus worried that it would lead to overfitting. Could this be mitigated by increasing the size of the time series?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ I don't see a reason not to try it, seems perfectly reasonable to me. Large network can work with high dimensional input (and indeed increasing the network capacity is good option) Note just to be rigorous and not to fall with some common pitfalls like snooping into the data and have window size so no training data will contain information about the output. $\endgroup$
    – ofer-a
    Commented Sep 11, 2021 at 13:50
  • $\begingroup$ There's a lot of literature about using machine learning and neural networks for stock market prediction. I'd start by conducting a literature review to see if anyone has tried this kind of a model before, and what they learned. $\endgroup$
    – Sycorax ♦
    Commented Sep 11, 2021 at 14:57
  • $\begingroup$ Except if you are doing that for purely academic purposes, if you use other firms opening price to forecast the one you are interested in, how are you going to use those forecasts? $\endgroup$
    – Firebug
    Commented Sep 13, 2021 at 14:19

1 Answer 1

Reset to default
0
$\begingroup$

There a whole bunch of ways you could go about this. Overfitting is definitely a danger if you have too many features. A few suggestions:

  • Rather than using all of the other stocks in the sector and potentially exploding the number of features in your dataset, try constructing some indicator features (e.g. average historic returns in sector, average volatility in sector, average historic returns of all stocks, fraction of stocks with positive returns). Hand crafting these will probably be your best return on investment in terms of performance.
  • Clustering approaches (kmeans etc.) are good too, they can help you find clusters of stocks which are positively (or negatively) correlated with the stock you want to predict.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.