3
$\begingroup$

I got an interview question:

Providing 1 year monthly data (12 points), can be traffic, product consumption, etc. How to forecast next month?

I am confused. This question doesn't looks like a time series question, because only 12 points are provided. If I just answer using January data in the last year as January data of this year, plus average yearly trend movement, is it too simple?

$\endgroup$
  • $\begingroup$ Your question (and I realize the question isn't actually yours) seems to be unclear. There's not nearly enough information to suggest a model, or to judge if a model might be too simple. With so little data, subject knowledge (how that particular kind of data tends to behave) becomes critical. $\endgroup$ – Glen_b Mar 22 '15 at 5:48
  • $\begingroup$ Thanks,that is what I thought. I think the interviewer didn't think clearly. $\endgroup$ – lserlohn Mar 22 '15 at 6:47
  • $\begingroup$ related question: stats.stackexchange.com/questions/135061/… $\endgroup$ – Tim Mar 22 '15 at 10:35
5
$\begingroup$

There's not nearly enough information to suggest a model, or to judge if a model might be too simple. With so little data, subject knowledge (how that particular kind of data tends to behave) becomes critical.

In an interview, you might take the strategy of suggesting several potential models - "If there's expected to be strong seasonality, and not a strong trend, maybe you could do this; if strong seasonality and strong trend seem likely, maybe do that; if seasonality and trend would be expected to be weak and noise high, perhaps do this ..." and so on.

(Though if it were me, I'd narrow it down finer than that.)

One might then say something like "if we really don't know what it is we're dealing with, and with little data, very simple models tend to forecast better than complex ones; perhaps exponential smoothing or double exponential smoothing might be one choice if we don't have more indication of what kind of model might be suitable."


(Added later in response to the request in comments)

As support for the claim that "very simple models tend to forecast better than complex ones" (particularly with little data), see for example, Makridakis and Hibon (2000) [1], discussing Makridakis and Hibon (1979) [2]:

The major conclusion of the Makridakis and Hibon study was that simple methods, such as exponential smoothing, outperformed sophisticated ones.

The statement was controversial in 1979, but results from the subsequent M-competitions broadly supported that conclusion (though the statements became somewhat more nuanced); similar sentiments can be found (for example) in the forecasting book by Makridakis, Wheelwright and Hyndman.

More broadly, see (for example) Green and Armstrong (2016) [3]:

Our review of studies comparing simple and complex methods — including those in this special issue — found 97 comparisons in 32 papers. None of the papers provide a balance of evidence that complexity improves forecast accuracy. Complexity increases forecast error by 27 percent on average in the 25 papers with quantitative comparisons. The finding is consistent with prior research to identify valid forecasting methods: all 22 previously identified evidence-based forecasting procedures are simple

[More recently, averages of forecasts have in many cases been found to perform quite well, but again those model-average forecasts have often tended to average over fairly simple models]

[1] Makridakis, S and Hibon, M (2000).
"The M-3 Competition: results, conclusions, and implications".
International Journal of Forecasting, 16 (October–December), 451-476

[2] Makridakis, S., & Hibon, M. (1979).
Accuracy of forecasting: an empirical investigation (with discussion).
Journal of the Royal Statistical Society A, 142, 97–145

[3] Kesten C. Green, K.C., and Armstrong, J. S. (2015),
Simple versus complex forecasting: The evidence
March 1, 2015
(forthcoming in Journal of Business Research)
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2643534

$\endgroup$
  • $\begingroup$ Do you happen to have a source for your last quote/statement? Not because I don't believe you, but because I'd like to use the quote of a verifiable source. $\endgroup$ – Grafit Mar 8 '16 at 10:56
  • $\begingroup$ @Grafit do you specifically mean "simple models tend to forecast better than complex ones"? I've added some stuff on that to the answer. $\endgroup$ – Glen_b Mar 8 '16 at 11:11
  • 1
    $\begingroup$ Overfitting of the data often implies poorer performance outside the sample, which is key here. $\endgroup$ – Nick Cox Mar 8 '16 at 11:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.