1
$\begingroup$

Consider the linear model $Y = B_{0} + B_{1}X_{1} + B_{2}X_{2} + e$, where the columns $X_{1}$ and $X_{2}$ of the design matrix have mean $0$ and length $1$. That is $X_{i}'X_{i} = 1$ and $X_{i}'J = 0$ where $J$ is a column entirely of ones. Let $\rho$ be the correlation between $X_{1}$ and $X_{2}$.

Q: Determine what values of $\rho$ will make the variance of $\hat{B}_1$ and $\hat{B}_2$ larger than $5\sigma^2$.

A: My thought is that this is a question related to the variance inflation factor. Therefore, I could plug in $1 / (1 - z^2) = 5$. Here, $z$ is the correlation I am trying to solve for. After performing a few algebra steps, I get $z = 0.8$ (Correlation). Please let me know if I am on the right track with the variance inflation factor or if I am supposed to use another methodology (covariance matrix, etc.) Thank you.

$\endgroup$
3
  • $\begingroup$ How are you relating the VIF to $5\sigma^2$? (BTW, you can mark up the math with $\TeX$ by enclosing it in dollar signs \$.) $\endgroup$
    – whuber
    Jul 7 '13 at 20:15
  • $\begingroup$ I have no idea.. I'm stabbing in the dark. How am I supposed to increase the variance by 5 of the variance?!? Seems like circular logic. $\endgroup$
    – simplemts
    Jul 7 '13 at 20:32
  • $\begingroup$ It looks like whoever asked you this question assumes you know, or have access to, a formula for the covariance matrix of the estimates $(\hat{B}_0, \hat{B}_1, \hat{B}_2)$. That formula probably is in terms of a multiple of the variance of $e$, which is what I presume "$\sigma^2$" refers to. That gives you an explicit connection you can exploit. $\endgroup$
    – whuber
    Jul 7 '13 at 20:38
1
$\begingroup$

The variance of $\hat{B}_1$ is $$ {\rm var}(\hat{B}_1) = \sigma^2 ([X^{'}X]^{-1})_{22}, $$ where $\sigma^2 = {\rm var}(e)$, and $X = [\boldsymbol{1}, X_1, X_2]$ denotes the design matrix (see for example here). From this question, we have $$\begin{align} X'X & = \begin{pmatrix} n & 0 & 0 \\ 0 & 1 & \rho \\ 0 & \rho & 1 \end{pmatrix}, \end{align} $$ where $n$ denotes the number of observations, and hence $$ [X^{'}X]^{-1} = \frac{1}{n (1 - \rho^2)} \begin{pmatrix} (1 - \rho^2) & 0 & 0 \\ 0 & n & -n \rho \\ 0 & -n \rho & n \end{pmatrix} . $$ Therefore, $$ {\rm var}(\hat{B}_1) = \frac{\sigma^2}{ (1 - \rho^2)}. $$ So, ${\rm var}(\hat{B}_1) > 5 \sigma^2$ if and only if $|\rho| > \sqrt{0.8}$. The result is the same for $\hat{B}_2$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.