3
$\begingroup$

I am trying to prove that an equivalent way to perform the test for significance of regression in multiple linear regression is to base the test on $R^2$ as follows:

to test $H_0:\beta_1 = \beta_2 = ... = \beta_k$ versus $H_1$ : at least one $\beta_j \neq 0$, calculate $$F_0=\frac{R^2(n-p)}{k(1-R^2)}$$ and to reject $H_0$ if the computed value of $F_0$ exceeds $F_{\alpha,k,n-p}$, where $p = k + 1$.

Now, I am starting from the easier case of simple linear regression, and so I want to show that $$t^2=\frac{R^2(n-2)}{1-R^2}.$$ I did the following. $$t=\frac{\hat{\beta_1}}{se(\hat{\beta_1})}=\frac{S_{xy}}{S_{xx}}\sqrt{\frac{S_{xx}(n-2)}{SS_{res}}} \Rightarrow t^2= \frac{S^2_{xy}(n-2)}{S_{xx}SS_{res}}$$ and $$R^2=\frac{SS_{reg}}{SS_{tot}}=\frac{S^2_{xy}}{S_{xx}SS_{tot}} \Rightarrow 1-R^2=\frac{S_{xx}SS_{tot} - S^2_{xy}}{S_{xx}SS_{tot}} \Rightarrow \frac{R^2(n-2)}{1-R^2}=\frac{S^2_{xy}(n-2)}{S_{xx}SS_{tot} - S^2_{xy}}.$$

How do I conclude? Thanks!

$\endgroup$
1
  • $\begingroup$ Welcome - would you maybe consider rethinking the title of your question? I feel it could better reflect your actual topic/question $\endgroup$ Mar 17 at 7:20
2
$\begingroup$

Notation:

$$ S_{xy}=\sum_{i=1}^{n}(x_i-\bar x)(y_i-\bar y) $$ and correspondingly for $S_{xx}$. $SS_{tot}=\sum_{i=1}^{n}(y_{i}-\bar{y})^{2}$, $SS_{reg}=\sum_{i=1}^{n}(\widehat{y}_{i}-\bar{y})^{2}$ and $SS_{res}=\sum_{i=1}^{n}\hat{u}_{i}^{2}$ stand for total (i.e., variation of the $y_i$), regression (i.e., variation of the fitted values) and residual sum of squares.

We have that $$ R^2=\frac{S_{xy}^2}{S_{xx}SS_{tot}} $$ and, from $SS_{tot}=SS_{reg}+SS_{res}$, $$ R^2=1-\frac{SS_{res}}{SS_{tot}} $$ so that $$ SS_{tot}(1-R^2)=SS_{res}^2 $$ and hence $$ t^2= \frac{S^2_{xy}(n-2)}{S_{xx}SS_{tot}(1-R^2)}. $$ One could also have continued from your denominator $S_{xx}SS_{tot} - S^2_{xy}$ to get $$ S_{xx}(SS_{reg}+SS_{res}) - S^2_{xy}, $$ so that it would remain to show $$S_{xx}SS_{reg} - S^2_{xy}=0.$$ This follows from $$ SS_{reg}=\frac{S^2_{xy}}{S_{xx}}, $$ which, in turn, follows from $$\widehat{y}_{i}=a+bx_{i}$$ and $$\widehat{y}_{i}-\bar{y}=b(x_{i}-\bar{x})$$ so that $$SS_{reg}=b^{2}\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}=b^{2}S_{xx}.$$ Plugging in $$b=S_{xy}/S_{xx}$$ gives $$SS_{reg}=\frac{S^2_{xy}}{S_{xx}}$$

$\endgroup$
3
  • $\begingroup$ Very nice! Thanks! $\endgroup$
    – mat95
    Mar 16 at 16:09
  • $\begingroup$ I would like to understand this. What is S? And is SS equivalent to S*S? $\endgroup$
    – KaPy3141
    Mar 17 at 8:51
  • $\begingroup$ Indeed, notation was explicit in this thread so far. I made an edit $\endgroup$ Mar 17 at 9:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.