8
$\begingroup$

I'm trying to forecast Poisson data, divided in groups, of 1-26 months of data, depending on the group. Of the pooled data 65% has a value of 0 and 25% a value of 1. I couldn't find any trends or seasonality, so I started to test a couple of different stationairy models. Moving average (3), Moving Average (6), Simple Exponential Smoothing, Naïve and Simple Mean.

I need to forecast 1-6 months ahead and used MAD, MSE and RMSE to test the accuracy of the models. It looks like the most accurate is Simple Mean, with an RMSE of 1 and an MAD of 0,638. I think this is really high but I have no clue how to do anything about this.

Are there forecasting methods I didn't think about that could be way better? Am I over-looking something?

The only thing I was able to find about prediction intervals was F+ts and F-ts with F as forecast, t as t distribution with alfa (n-2) and s as standard deviation. It don't think it was a really trustworthly source but since I wasn't able to find anything else, I'm not sure about how to set up those prediction intervals. Is this method right?

I don't have R to use. I need to do it myself.

$\endgroup$
10
$\begingroup$

You have what is called intermittent demand, that is, a demand time series characterized by "many" zeros. (If your time series is not demand per se, most of what follows will still apply.) So a web search for "forecasting intermittent demand" would already be helpful. Teunter and Duncan (2009, JORS) give an overview of intermittent demand forecasting methods.

The standard method of forecasting intermittent demands is Croston's method. Use exponential smoothing on inter-demand intervals and on nonzero demand sizes separately. The point forecast then is the ratio of the smoothed nonzero demand to the smoothed inter-demand interval. Syntetos and Boylan (2001, IJPE) note that Croston is slightly biased and propose a modification, but this usually doesn't make all that much of a difference in practice.

An alternative is integer autoregressive moving average models (INARMA), which modify the standard ARIMA time series models. Maryam Mohammadipour wrote a thesis on these.

I personally have major doubts about the usefulness of such an expectation point forecast. A time series of 1 demand every other time period has an expectation of 0.5... as does a time series of 2 demands every fourth time period... and so forth - although these are, of course, less and less Poisson-y. I'd argue that it's much more useful to understand the entire future (and predictive) distribution of demands. So I applaud your looking for prediction intervals!

However, the $\alpha(n-2)$ formula you found applies only to single exponential smoothing on continuous data, via the ARIMA model SES is optimal for. So it is inapplicable to count data. I'd much rather propose that you take your point prediction $\hat{y}$ and use quantiles of the Poisson distribution with parameter $\lambda=\hat{y}$. This still disregards parameter estimation uncertainty (along with model selection uncertainty etc.), but it's a simple possibility and likely better than the formula you have.

Shenstone and Hyndman (2005, JoF) note that there is no consistent stochastic model for which Croston's method would be optimal - all candidate models are (1) continuous, not discrete, and (2) can yield negative values. However, for those candidate models, Shenstone and Hyndman provide prediction intervals.

Finally, a word of caution: don't use the MAD for assessing the accuracy of count data forecasts, especially not for intermittent demands. The expected MAD is minimized by the median of your future distribution, not its mean, and if you write that 65% of your data are zeros, then the median is zero... implying that you will probably get the lowest MAD by a flat zero forecast, which is badly biased and likely useless. Here is a presentation I gave at last year's International Symposium on Forecasting on this issue. Or look at Morlidge (2015, Foresight).

Final piece of shameless self-promotion: I have an article in the IJF (Kolassa, 2016) which looks at forecasting low volume count data (mostly intermittent), at different accuracy measures and different forecasting methods, including various flavors of Poisson models. This may be useful to you.

$\endgroup$
  • 1
    $\begingroup$ Thanks for the reaction Stephan. You give me a lot of new knowledge and insights, I'm completely turning my ways around. I would very much like a PDF of your manuscript, I think it would be really helpful $\endgroup$ – R. White Mar 24 '16 at 12:27
  • 1
    $\begingroup$ Another question, if you don't mind. this article describes how I should test for intermittency in my data. Now I notice a lot of the medians of qi per group have a value of 1. Lower isn't really possible so that would assume most of my groups aren't intermittent, right? $\endgroup$ – R. White Mar 24 '16 at 16:22
  • 1
    $\begingroup$ That classification is new to me. It's certainly not common in the academic literature. However, I know the authors of that SAS paper, and they usually know what they are doing. I'd recommend that you try intermittent and non-intermittent methods on your data and see which works best - then see whether that has anything to do with the criteria given in the SAS paper. Syntetos & Boylan (2005) and Boylan et al. (2008) give alternative classifications. $\endgroup$ – S. Kolassa - Reinstate Monica Mar 24 '16 at 20:13
  • 3
    $\begingroup$ Intermittent demands are hard to forecast. The problem is that the difference between zero demand and nonzero demand is (relatively!) large - and that we usually don't know when the nonzero demand will occur. I suggest that you run a few simulations. Simulate Poisson random variables and check what RMSE you get when you forecast the (known!) expectation. This simulates the situation where you know how high demand is on average, but you don't know when the demand will occur. You will probably find that RMSEs are pretty large. $\endgroup$ – S. Kolassa - Reinstate Monica Apr 7 '16 at 11:54
  • 2
    $\begingroup$ Bottom line: unless you can forecast when demand will occur, intermittent demands are just hard to forecast. In addition, if you forecast that demand will occur in period $t$, but it does occur in period $t+1$, that may be quite OK for your forecast-consuming processes - but RMSE and other error measures will still count this as a Fail .There have been ideas by Kourentzes (2014 - cited in my paper) for error measures relying on cumulative forecasts and demands that penalize this kind of "timing error" less strongly than larger "timing errors". $\endgroup$ – S. Kolassa - Reinstate Monica Apr 7 '16 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.