I've used a handful of algorithms for forecasting future values in a time series. But sometimes what I'm really interested in is not the predicted value, but the probability that some future will be below (or above) some threshold.

So I'm trying to figure out a way to turn the output of a forecast into an estimate of probability. Forecast algorithms usually return, at minimum, a prediction and a confidence interval (often 95%). So perhaps I could make the assumption that the probability density function is normally distributed, centred on the prediction.

If $x_{pred}$ is the prediction, the probability density would be

$f(x) = \frac{1}{\sqrt{2\pi\sigma^2} } e^{ -\frac{(x-x_{pred})^2}{2\sigma^2} }$

So next I need to know (or estimate) $\sigma$. The 95% confidence interval is

$x_{pred} \pm 1.960 \frac{\sigma}{\sqrt{n}}$,

so I can calculate $\sigma$ if I know (or can estimate) n. Would it make any sense to set $n \approx$ the number of data points used in making the forecast?

Alternatively, forecast models often provide the standard error. Would it make any sense to use this as an estimate of $\sigma$?

In theory, I could re-implement a standard algorithm to get access to the information I need. However, I want to be able to use existing implementations, including ones that might be available in future.

  • 1
    $\begingroup$ I would definitely have a look at Gaussian Processes. They do exactly those things that you describe: the predictive distribution is a normal distribution with parameters influenced by the current x input and the training data. See en.m.wikipedia.org/wiki/Gaussian_process, m.youtube.com/watch?v=vU6AiEYED9E and so forth... $\endgroup$ – Fabian Werner Jun 19 '19 at 11:54
  • $\begingroup$ You are using the term confidence interval where prediction interval is more appropriate. $\endgroup$ – Richard Hardy Jun 19 '19 at 14:23
  • $\begingroup$ stats.stackexchange.com/questions/410806/… will be of help to you. $\endgroup$ – IrishStat Jun 19 '19 at 14:51
  • $\begingroup$ @FabianWerner that looks like exactly what I need. If you want to turn that into an answer, I'll accept it. $\endgroup$ – mhwombat Jun 20 '19 at 10:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.