In product demand forecasting, is it valid to use winsorization to remove large outliers (spikes) in the data? I understand that the spikes may be due to holiday effects (e.g. people will buy more chocolate during Valentine's day). However, they seem to be skewing the data.

The approach that I intend to adopt is that by Neal Wagner in Intelligent techniques for forecasting multiple time series in real-world systems in which explainable spikes are first removed, then added back in after the modeling task. The paper states that spikes are "removed", but how they are removed is never expounded upon. In my case, I intend to use winsorization.

  • $\begingroup$ Holiday effects and seasonality are not generally thought of as "outliers"; they are usually explicitly modeled. Do you have "spikes" for which you have absolutely no pattern/explanation? For those, your model can incorporate effects related to these outliers after it has seen them, but you should not first remove them for the entire series, then measure your pseudo-out-of-sample forecast accuracy on the already adjusted data (that would be cheating). Is that the approach you're suggesting? $\endgroup$ – Chris Haug May 1 '18 at 23:29
  • $\begingroup$ Yes, that was what was suggested to me, but it did seem invalid, which is why I doubled checked it here. For spikes that seem spurious and not explainable, how do I go about "incorporating effects related to them" as you state? $\endgroup$ – meraxes May 2 '18 at 3:56
  • $\begingroup$ Why do they seem inside invalid? Are the outlets on holidays or not? In your question you said they are. $\endgroup$ – Tim May 2 '18 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.