5
$\begingroup$

A known result states that if $ A $ is a $ p \times p $ non-singular matrix, and we have the data matrix $ X : n \times p $ and the dependent variable vector $ Y : n \times 1 $, for which we have obtained the estimated coefficient vector $ \beta = (X^T X)^{-1} X^T Y $, then if we run OLS again with the transformed data matrix $ X^* = XA $, the new estimated coefficient vector $ \beta^* = ({X^*}^T X*)^{-1} {X^*}^T Y = A^{-1} \beta $.

Are there similar results for when we have a matrix $ A' : n \times n $, and we have transformed the dependent variable vector $ Y $ (i.e. using $ Y^* = A' Y $ in our regression)? Is there some literature that someone can point me to?

$\endgroup$
1
  • $\begingroup$ Results for constrained $ A' $ would also be interesting to me (e.g. all columns of $ A' $ must sum to 1). $\endgroup$
    – Kevin Li
    Jul 10, 2018 at 14:49

1 Answer 1

2
$\begingroup$

When you apply a linear transformation $A_{p \times p}$ to your explanatory variables stacked in $X_{n \times p}$, you are merely combining variables from one observation in your sample among themselves. $X * A$ does not mix variables between different observations, so, if we're both in the dataset, my observations are never mixed with yours. That is essential to the results stating, for instance, that you cannot improve the explained variance of $Y$ by applying a linear transformation $A$ to your dataset.

Moreover, the analogous transformation to $Y_{n \times 1}$ would be $A_{1 \times 1}$ which is of course only a real number that would rescale your data, thus rescaling your $\beta_{p \times 1} * A$ accordingly.

Another way of perceiving this question as a rather weird one is that in doing so with a $A_{n \times n}$ is that you would combine erros from one observation into another one. If you assume that errors are uncorrelated through observations (i.e., that $\Sigma = E[\varepsilon_{n \times 1} * \varepsilon_{n \times 1}']$ is diagonal), then you are inducing correlation between errors (the new covariance matrix will be $A \Sigma A'$).

There are cases in which you would like to use a properly chosen matrix $A_{n \times n}$ to combine your observations, but you would also apply this matrix $A_{n \times n}$ to your explanatory variables $X$. This is only the case when $\Sigma$ is already assumed not to be a constant variance diagonal matrix and then the matrix $A$ is chosen optimally to make $A \Sigma A'$ a diagonal and homoskedastic covariance matrix. You may find more about this looking for Generalized Least Squares.

Finally, note that if $A$ is a diagonal matrix, then you are merely rescaling each observation with different scale factors $a_{ii}$. This is a subcase of the previous paragraph and is a transformation that would multiply the variance of each observation's error by $a_{ii}^2$. Note that if $a_{ii} = 1/\sqrt{\sigma_i}$, then the variance of each new error term is equal to 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.