Holt-Winters and importance of R-square

Question

Is R-square an important measure in Holt-Winters method?

Answer 1

From my experience in forecasting: no.

The problem is that (as with any data analysis method) by increasing the complexity of the model, one can reduce R squared, even if the more complex model does not improve the predictive power of the model. In Exponential Smoothing, this would involve going from Single Exponential Smoothing to smoothing with a trend (Holt's method) to smoothing with trend and seasonality (Holt-Winters). In-sample fit and thus R squared will improve with each step in complexity - but forecasts may very easily get worse.

Conversely, it is hard to get a decent concept of "degrees of freedom" in smoothing, so calculating an "adjusted R squared" to account for this spurious improvement in in-sample fit is hard.

Bottom line: I have been forecasting for years now, but R squared is not something I see applied to smoothing often. Better to keep a holdout sample and compare forecasts for that holdout sample for different models.

Answer 2

Not usually. The main concern in forecasting is likely to be a measure of likely forecast error such as root mean squared error (or MAPE, MPE, or others).

Of course, with the original variance and the mean squared error of the forecast model it is easy to compute R squared, but that's usually not the emphasis.