The stock market value of the data point connected by the red line is predicted by linear regression using market values as well as Twitter sentiment data and more in a certain period of time (red area).

Ideally, this should result in some form of uncertainty quantification, i.e.: the further the model tries to predict from the known state of the market, the more uncertain the predicted value becomes.

Currently my model predicts the same change of market value no matter how far ahead we are trying to predict.

Here is an visualization of two data points: features from the red zone with the market value difference as label.

Vis 2 data points

What is a good way of finding some kind of uncertainty quantification depending on how far ahead we predict?

Adding a feature "days_ahead" to give some sort of prediction window seems too simplistic, because I do not believe that my model will understand the meaning of it. How can I incorporate this information?

  • 1
    $\begingroup$ Is there any way you can measure your model's error rate vs. the time lag on historical data? $\endgroup$ – jkm Oct 16 at 12:24
  • $\begingroup$ Yes, that is possible. I could "tune" lag as a parameter. But lags need to be variable. For example: a lot of negative tweets might result in a drop of market value within 2 days - but then tons of positive tweets could do the opposite in a 1 week window. $\endgroup$ – Laksan Nathan Oct 16 at 12:29
  • $\begingroup$ If you can measure residual variance of your model predicting a step ahead, it is possible to calculate the variance for n steps ahead. $\endgroup$ – Alexey says Reinstate Monica Oct 16 at 12:37

No matter how you slice it, this is effectively a different question - instead of looking for a value Y(X), you're asking about the parameters of a distribution with a mean of Y(X) and variance/standard deviation E(X, deltaT).

I see two broad approaches here:

1) The simpler implementation is to run your current model on some historical data, tweak the lag, then measure the error vs. lag value. If there's a nice trend, you can throw a separate regression on top to estimate the rate of increase in stddev given a delay.

You could then plug the lag you want to run your main model with into the new regression to get your uncertainty window, and use the main model to predict the mean.

The nice thing here is that you have zero impact on the performance of the value predictor, but you'd need to babysit the extra model to ensure its predictions make sense.

2) The more fanciful approach would be to go full Bayes, and have a single model predict both a value and its own uncertainty. You would need the delay as a feature of the model in that case.

This may be more complicated to explain, and may be less performant than a simple LinReg. Note that it may be actually easier to implement than (1), because you might find some out-of-the-box solution, while in (1) you need to create a whole extra model for the error and coordinate the two predictors yourself.

  • $\begingroup$ The separate regression from 1) predicts the expected error from the first model given only the delay/lag? $\endgroup$ – Laksan Nathan Oct 18 at 8:06
  • 1
    $\begingroup$ Yeah, it's a model assuming that the gradient of error with respect to time is constant and estimates that constant, so that you can just plug the lag into some polynomial when you want to do some estimatin'. $\endgroup$ – jkm Oct 18 at 8:16
  • $\begingroup$ Thanks a lot for your Input! $\endgroup$ – Laksan Nathan Oct 18 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.