Why can't we use top-down methods in forecasting grouped time series?

Question

As I asked in here I was trying to forecast grouped time series with two grouping variables and I find some limitation of hierarchical forecasting methods. In particular, using hts package from R, we can't use top-down methods.

I consider grouped time series which can be viewed as:

     Total
   |       | 
   A       B
 |   |    |   |
AX  AY   BX  BY

     Total
   |       | 
   X       Y
 |   |   |   |
 AX  BX  AY  BY

(It's described in more details in this post and for example in this paper)

According to the notation specified in this paper we can write such grouped time series as $\mathbf{Y_t} = \mathbf{S} \mathbf{Y_{K,t}}$, where $\mathbf{S}$ is a summing matrix and $\mathbf{Y_{K,t}}$ is a vector of bottom level series (which according to assumption in hts package have to be equal). In this case it looks like:

$$ \begin{bmatrix} Y_t \\ Y_{A,t} \\ Y_{B,t} \\ Y_{X,t} \\ Y_{Y,t} \\ Y_{AX,t} \\ Y_{AY,t} \\ Y_{BX,t} \\ Y_{BY,t} \\ \end{bmatrix} = \begin{bmatrix} 1 & 1 & 1 & 1 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 \\ 1 & 0 & 1 & 0 \\ 0 & 1 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{bmatrix} \begin{bmatrix} Y_{AX,t} \\ Y_{AY,t} \\ Y_{BX,t} \\ Y_{BY,t} \\ \end{bmatrix} $$

Revised forecast (what I am looking for) can be written as $\mathbf{\tilde{Y}_n(h) = SP\hat{Y}_n(h)}$ and in case of top-down method matrix $\mathbf{P}$ is defined as $\mathbf{P} = \begin{bmatrix} \mathbf{p} | \mathbf{0}_{m_K \times (m-1)} \end{bmatrix}$, where $ \mathbf{p} = [p_1, p_2, ..., p_{m_K}]^T$ is a vector of proportions. Not going into more details, in this example $m_K = 4$ and $m=9$, so $\mathbf{P} = \begin{bmatrix} \mathbf{p_1} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \mathbf{p_2} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \mathbf{p_3} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \mathbf{p_4} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \end{bmatrix}$

and revised forecasts can be written as:

$$ \begin{bmatrix} \tilde{Y_t} \\ \tilde{Y}_{A,t} \\ \tilde{Y}_{B,t} \\ \tilde{Y}_{X,t} \\ \tilde{Y}_{Y,t} \\ \tilde{Y}_{AX,t} \\ \tilde{Y}_{AY,t} \\ \tilde{Y}_{BX,t} \\ \tilde{Y}_{BY,t} \\ \end{bmatrix} = \begin{bmatrix} 1 & 1 & 1 & 1 \\ 1 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 \\ 1 & 0 & 1 & 0 \\ 0 & 1 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end{bmatrix} \begin{bmatrix} p_1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ p_2 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ p_3 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ p_4 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \end{bmatrix} \begin{bmatrix} \hat{Y_t} \\ \hat{Y}_{A,t} \\ \hat{Y}_{B,t} \\ \hat{Y}_{X,t} \\ \hat{Y}_{Y,t} \\ \hat{Y}_{AX,t} \\ \hat{Y}_{AY,t} \\ \hat{Y}_{BX,t} \\ \hat{Y}_{BY,t} \\ \end{bmatrix} $$

and after calculations:

$$ \begin{bmatrix} \tilde{Y_t} \\ \tilde{Y}_{A,t} \\ \tilde{Y}_{B,t} \\ \tilde{Y}_{X,t} \\ \tilde{Y}_{Y,t} \\ \tilde{Y}_{AX,t} \\ \tilde{Y}_{AY,t} \\ \tilde{Y}_{BX,t} \\ \tilde{Y}_{BY,t} \\ \end{bmatrix} = \begin{bmatrix} p_1\hat{Y_t} + p_2\hat{Y_t} + p_3\hat{Y_t} + p_4\hat{Y_t} \\ p_1\hat{Y_t} + p_2\hat{Y_t} \\ p_3\hat{Y_t} + p_4\hat{Y_t} \\ p_1\hat{Y_t} + p_3\hat{Y_t} \\ p_2\hat{Y_t} + p_4\hat{Y_t} \\ p_1\hat{Y_t} \\ p_2\hat{Y_t} \\ p_3\hat{Y_t} \\ p_4\hat{Y_t} \\ \end{bmatrix} $$

Which seems OK for me. I was hoping that somebody could point out why this method can't be used in forecasting grouped time series and point out when my calculations are wrong?

Answer 1

The top-down methods implemented in the hts package were designed for hierarchical time series. If you want to define your own top-down method for some non-hierarchical time series, go right ahead. It's not wrong, it just hasn't been implemented in the hts package because there are much better solutions to the problem.

The best approach currently available is to use weighted least squares as explained in http://robjhyndman.com/working-papers/hgts/. This is the default in the hts package.

Answer 2

Further to Rob's answer, let's look at your specific example:

     Total
   |       | 
   A       B
 |   |    |   |
AX  AY   BX  BY

     Total
   |       | 
   X       Y
 |   |   |   |
 AX  BX  AY  BY

You have not one, but two hierarchies. For instance, the top one may group sales at product-location level first by location A and B and then add these up to get the Total, and the second one may group first by product X and Y and add these up.

To do top-down forecasting, you forecast the Total and then break this down by proportions. You can use historical proportions, in which case, yes, you can do this.

The problem arises if you want to use forecasted proportions to break down the Total forecasts. Why? Because if you forecast your component series A, B, X, Y separately, these forecasts will usually not be sum consistent. That is, if you first break the Total down to A & B and then break A down to AX & AY, you may get a different result for AX than if you first broke down to X & Y and then broke X down to AX & BX.

Thus, you will need to do some kind of reconciliation. And then, as Rob writes, you might as well use the "real" optimal reconciliation approach.

(Note that this problem does not arise if you use historical proportions to break forecasts down, because these are automatically consistent. However, historical proportions do not allow you to forecast out changing dynamics in your hierarchy, like changing market shares.)