Given a simple linear model:

$$ y_i = x_i^T \beta + \epsilon_i $$

For simplicity, $\epsilon_i$ is Gaussian iid with variance $\sigma_e^2$, then the solution for $\hat{\beta}$ is given via Ordinary Least Squares:

$$ \hat{\beta} = (X^T X)^{-1} X^T y $$

Then it can be shown that the MSE is given as:

$$ \mathrm{MSE} = \operatorname{Var}(\hat{\beta} - \beta) = \frac{1}{L} \| \hat{\beta} - \beta \|_2^2 = \frac{1}{L} \sigma_e^2 \operatorname{Tr} (X^T X)^{-1} $$

where $L$ is the number of unknowns (length of vector $\beta$). That means the MSE for the estimated coefficients can be represented using the trace of the covariance matrix!

Now suppose I pick a new $X'$ (which has the same staistics as $X$) and want to compare how good my estimator prediction is for a new dataset. I want to represent it using the normalized mean squared prediction error as follows:

$$ y' = X' \beta \\ \hat{y}' = X' \hat{\beta} \\ \mathrm{NMSPE} = \frac{\|\hat{y}'-y'\|_2^2}{\|y'\|_2^2} $$

Is there an elegant way to describe the NMSPE in a similar way as the MSE as a function of the covariance matrix (or similar)?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.