1
$\begingroup$

I read a paper which briefly mentions that the Mean Squared Error (MSE) of Vector Autoregression increases with the dimensionality. In detail, let $Y_t$ an $k$-dimensional process which I want to estimate using a VAR(p) model, i.e. $$Y_t = A_1 Y_{t-1} + \dots + A_p Y_{t-p} + u_t$$ where $u_t$ is a $k$-dimensional white noise error term. Let us estimate the coefficients with multivariate least squares and receive the estimated values $\hat{Y}_t$. How is the MSE with respect to $\hat{Y}_t - Y_t$ of this model related to $k$?

$\endgroup$
  • $\begingroup$ Hi Larry, how is my answer? Is it clear, or do you need further elaboration? $\endgroup$ – Richard Hardy Apr 6 '18 at 17:32
  • $\begingroup$ Dear Richard, thanks for your answer. I understand what you mean. In the meantime, I found a result with respect to the MSE matrix, which is according to Lütkepohl (New Introduction to Multiple Time Series Analysis, Ch. 3.5.2, 2007) indeed positively dependent on k. I think then the overall MSE (written as a one-dimensional sum) should be then also positively depend on k. $\endgroup$ – ReinerZufall Apr 9 '18 at 8:02
  • $\begingroup$ Larry, I am wondering if the answer is satisfactory or whether it needs some elaboration. FYI, satisfactory answers may be accepted by clicking on the tick mark to the left; this is how Cross Validated works. $\endgroup$ – Richard Hardy Jun 1 '18 at 20:08
1
$\begingroup$

Consider a vector $y$ of dimension $k$ and its fitted value $\hat y$. The squared error (SE) is $$ \text{SE}(y)=\sum_{i=1}^k (y_i-\hat y_i)^2 $$ where $y_i$ is the $i$-th component of $y$ and $\hat y_i$ is the $i$-th component of $\hat y$.

Now add some more dimensions to $y$, i.e. increase the dimension from $k$ to $k+h$ for some integer $h>0$. Name the variable $z$ and its fitted value $\hat z$. The first $k$ elements of $z$ constitute $y$ but the last $h$ elements are new. If the fitted value of the first $k$ elements of $z$ is $\hat y$, you get \begin{aligned} \text{SE}(z) &= \sum_{i=1}^{k+h} (z_i-\hat z_i)^2 \\ &=\sum_{i=1}^k (z_i-\hat z_i)^2+\sum_{i=k+1}^{k+h} (z_i-\hat z_i)^2 \\ &=\sum_{i=1}^k (y_i-\hat y_i)^2+\sum_{i=k+1}^{k+h} (z_i-\hat z_i)^2 \\ & \geq\sum_{i=1}^k (y_i-\hat y_i)^2 \end{aligned} simply because each square in these sums is nonngegative.

This is for one realization of $y$ and $z$ and their fitted values. If you have multiple realizations and their corresponding fitted values, you would take a simple average of $\text{SE}(y)$ vs. a simple average of $\text{SE}(z)$, i.e. $\text{MSE}(y)$ vs. $\text{MSE}(z)$. Since each $\text{SE}(z)$ is at least as great as the corresponding $\text{SE}(y)$, the average (the $\text{MSE}$) will also be at least as great.

This holds if the fitted value of $y$ is the same as the fitted value of the first $k$ elements of $z$. (The realizations are exactly equal by the definitions of $y$ and $z$, but the fitted values need not be since they are generated by different VAR models.) But if the fitted value accuracy for particular dimensions increases much when the overall dimension is increased (i.e. the new $h$ dimensions help predict the old $k$ dimensions very well and the new dimensions themselves are not too hard to predict), then it is possible that $\text{MSE}(z)<\text{MSE}(y)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.