0
$\begingroup$

Given RMSE skill score $s$: \begin{equation}\label{eq:msess} s = 1-\frac{\text{RMSE}(f,x)}{\text{RMSE}(r,x)}, \end{equation} where $f$, $r$, and $x$ are forecasts of interest, reference forecasts, and observations, respectively. Suppose $x$ has a known multiplicative seasonal component $c$ (which is strictly positive), i.e., $x = c\times k$, and $k$ is the deseasonalized quantity, and assume reference forecasts are generated using the seasonally adjusted persistence model, the RMSEs can thus be written as: \begin{eqnarray} \text{RMSE}(f,x) &=& \sqrt{\frac{1}{N}\sum_{t=1}^{N}\left(f_{t}-x_t\right)^2} \\ \text{RMSE}(r,x) &=& \sqrt{\frac{1}{N}\sum_{t=1}^{N}\left(x_{t-1}\cdot c_t/c_{t-1}-x_t\right)^2}. \end{eqnarray} When does the skill score $s$ equal to $s'$, namely, the skill score computed based on the forecasts of the deseasonalized quantity? \begin{equation} s' = 1-\frac{U}{V}, \end{equation} where \begin{eqnarray} U &=& \sqrt{\frac{1}{N}\sum_{t=1}^{N}\left(\frac{f_t-x_t}{c_t}\right)^2},\\ V &=& \sqrt{\frac{1}{N}\sum_{t=1}^{N} \left(\frac{x_{t-1}}{c_{t-1}}-\frac{x_t}{c_t}\right)^2}. \end{eqnarray}


My current approach is to assume the seasonal component has $m$ discrete states, namely, $c^{(1)}$, $\cdots$, $c^{(m)}$, so that $U^2$ can be rewritten as: \begin{eqnarray} U^2 & = & \frac{1}{N}\left\{ \sum_{t\in\mathcal{N}_1}\left(\frac{f_t-x_t}{c^{(1)}}\right)^2 + \sum_{t\in\mathcal{N}_2}\left(\frac{f_t-x_t}{c^{(2)}}\right)^2 + \cdots + \sum_{t\in\mathcal{N}_m}\left(\frac{f_t-x_t}{c^{(m)}}\right)^2 \right\} \\ &=& \frac{1}{N}\left\{\frac{1}{\left(c^{(1)}\right)^2} \sum_{t\in\mathcal{N}_1}\left(f_t-x_t\right)^2 + \frac{1}{\left(c^{(2)}\right)^2}\sum_{t\in\mathcal{N}_2}\left(f_t-x_t\right)^2 + \cdots + \frac{1}{\left(c^{(m)}\right)^2}\sum_{t\in\mathcal{N}_m}\left(f_t-x_t\right)^2 \right\} \\ &=& \frac{1}{N}\left\{\frac{|\mathcal{N}_1|}{\left(c^{(1)}\right)^2} \mathbb{E}\left[(f-x)^2\big| c=c^{(1)}\right] + \frac{|\mathcal{N}_2|}{\left(c^{(2)}\right)^2} \mathbb{E}\left[(f-x)^2\big|c= c^{(2)}\right] + \cdots + \frac{|\mathcal{N}_m|}{\left(c^{(m)}\right)^2} \mathbb{E}\left[(f-x)^2\big| c=c^{(m)}\right] \right\}, \end{eqnarray} where $\mathcal{N}_1$, $\cdots$, $\mathcal{N}_m$ are the sets of data index that satisfy events $c_t=c^{(1)}$, $\cdots$, $c_t=c^{(m)}$, respectively. The notation $|\mathcal{N}_1|$ denotes the cardinality of $\mathcal{N}_1$. If the expected squared error of forecast $f$ is same for all $c$, i.e., \begin{equation} \mathbb{E}\left[(f-x)^2\big| c = c^{(1)}\right] = \mathbb{E}\left[(f-x)^2\big| c = c^{(2)}\right] =\cdots = \mathbb{E}\left[(f-x)^2\big| c = c^{(m)}\right] = \mathbb{E}\left[(f-x)^2\right] , \end{equation} then \begin{equation} U^2 = \frac{1}{N}\left\{\frac{|\mathcal{N}_1|}{\left(c^{(1)}\right)^2} + \frac{|\mathcal{N}_2|}{\left(c^{(2)}\right)^2} + \cdots \frac{|\mathcal{N}_m|}{\left(c^{(m)}\right)^2} \right\}\mathbb{E}\left[(f-x)^2\right]. \end{equation} Similarly, the $V^2$ can be written as: \begin{equation} \notag V^2 = \frac{1}{N}\left\{\frac{|\mathcal{N}_1|}{\left(c^{(1)}\right)^2} + \frac{|\mathcal{N}_2|}{\left(c^{(2)}\right)^2} + \cdots \frac{|\mathcal{N}_m|}{\left(c^{(m)}\right)^2} \right\}\mathbb{E}[\left(r-x\right)^2], \end{equation} if the squared reference forecast error, $(r-x)^2$, is independent of the seasonal component. As a result, \begin{eqnarray} s' &= & 1-\frac{\sqrt{ \frac{1}{N}\left\{\frac{|\mathcal{N}_1|}{\left(c^{(1)}\right)^2} + \frac{|\mathcal{N}_2|}{\left(c^{(2)}\right)^2} + \cdots \frac{|\mathcal{N}_m|}{\left(c^{(m)}\right)^2} \right\}}\sqrt{\mathbb{E}\left[(f-x)^2\right]} }{\sqrt{ \frac{1}{N}\left\{\frac{|\mathcal{N}_1|}{\left(c^{(1)}\right)^2} + \frac{|\mathcal{N}_2|}{\left(c^{(2)}\right)^2} + \cdots \frac{|\mathcal{N}_m|}{\left(c^{(m)}\right)^2} \right\}} \sqrt{\mathbb{E}\left[(r-x)^2\right]}} \\ \notag &= &1-\frac{\sqrt{\mathbb{E}\left[(f-x)^2\right]} }{ \sqrt{\mathbb{E}\left[(r-x)^2\right]}} \\ &=& 1-\frac{\text{RMSE}(f,x)}{\text{RMSE}(r,x)} = s, \end{eqnarray} if both the squared forecast error and squared reference forecast error are independent of the seasonal component. I'm not sure if this approach is valid.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.