I am forecasting demand using the Holt-Winters model for a particular product class.
I have been examining its performance so far this quarter (I only ever forecast Q4), and was surprised to note that the average value of the daily estimator is pretty much the same as the average observed value - so the forecast level is pretty much spot on (so far).
Additionally, the intra-year seasonality seems to be being captured broadly correctly, in the sense that the values go up and down on the correct days (there is a cycle of days within weeks, and weeks within the year - both are fairly predictable).
However, I noticed that the model is wildly exaggerating the swings "up" on the historical "up" weekdays, and "down" on the historical "down" weekdays. I at first thought that this was an issue to do with parameterisation or the choice of starting values, but I investigated and discovered that the actual culprit is that, although the seasonal cycle has remained constant in terms of which days are up and which are down (for the most part), the degree of swing in each direction is a lot less on each day this quarter than it was in Q4 2015. (I have compared a series of ti-ti-1 for both years). In other words, the seasonal cycles are the same in terms of direction each day, but not magnitude.
As (again, so far) the pattern has been consistently different, I have simply built a really simple algorithm in excel to cap the increases/decreases on peak/trough days to the percentage-terms increases/decreases which have been observed over this year's Q4 sample period. This actually works perfectly and reduces the mean absolute percentage error from 10.93%, with max 29%, down to 6.48%, with max 11.02%.
However, I'm pretty sure that this has absolutely zero statistical validity, and I'm not sure how meaningful it is that this dramatically improves performance over the past 55 days. In particular, how do I know whether the ratio of the seasonal-cycle amplitude this year to last year will be constant over the whole of the quarter, or whether it is a temporary phenomenon?
I'm sure that it would probably be possible to modify the seasonal-cycle amplitude by manipulating the parameters in the model so that it complied with the reality, but I think that this would just be equally arbitrary.
Of course, ideally, I should have data for all the exogenous variables which determine the variation and build a model which incorporates parameters for these, but I only have unitary time series data with no exogenous data available.
One final complicating factor - I have taken the series of absolute values of day-to-day percentage "swings" from this year, and analysed them alongside the equivalent values from the corresponding days last year. I have noticed that the gap between the two has been declining over the course of this year's sample period, and that the two appear to be gradually converging. But I don't know, again, whether this is meaningful or useful.