0
$\begingroup$

Given a simple linear model:

$$ y_i = x_i^T \beta + \epsilon_i $$

For simplicity, $\epsilon_i$ is Gaussian iid with variance $\sigma_e^2$, then the solution for $\hat{\beta}$ is given via Ordinary Least Squares:

$$ \hat{\beta} = (X^T X)^{-1} X^T y $$

Then it can be shown that the MSE is given as:

$$ \mathrm{MSE} = \operatorname{Var}(\hat{\beta} - \beta) = \frac{1}{L} \| \hat{\beta} - \beta \|_2^2 = \frac{1}{L} \sigma_e^2 \operatorname{Tr} (X^T X)^{-1} $$

where $L$ is the number of unknowns (length of vector $\beta$). That means the MSE for the estimated coefficients can be represented using the trace of the covariance matrix!

Now suppose I pick a new $X'$ (which has the same staistics as $X$) and want to compare how good my estimator prediction is for a new dataset. I want to represent it using the normalized mean squared prediction error as follows:

$$ y' = X' \beta \\ \hat{y}' = X' \hat{\beta} \\ \mathrm{NMSPE} = \frac{\|\hat{y}'-y'\|_2^2}{\|y'\|_2^2} $$

Is there an elegant way to describe the NMSPE in a similar way as the MSE as a function of the covariance matrix (or similar)?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.