4
$\begingroup$

I am trying to understand the relations between the coefficient of determination $R^2$ and the $F$-distributions.

Hereafter are the notations I use :

Assume a simple linear regression model $y_i=ax_i+b + \varepsilon_i, 1\leq i\leq n$, where the errors $\varepsilon_i$ follow a normal distribution with mean $0$ and constant variance $\sigma^2$. The regression line calculated by OLS is $y=\hat{a}x+\hat{b}$ and $\hat{y}_i=\hat{a} x_i +\hat{b}$ is the predicted value (at point $x_i$) and $\hat{\varepsilon}_i=y_i-\hat{y}_i$ is the residual.

Let $SSR=\sum(\hat{y}_i - \bar{y})^2$ be the sum of squares explained by the regression, $SSE=\sum(y_i - \hat{y}_i)^2$ the sum of squared errors and $SST=\sum(y_i - \bar{y})^2$ the total sum of squares.

Then $R^2=\frac{SSR}{SST}=1-\frac{SSE}{SST}$.

What I understood from documents about linear regression is that, under the null hypothesis that $a=0$ (the slope of the true regression line is zero), the two statistics $ \frac{SSR}{\sigma^2}$ and $ \frac{SST}{\sigma^2}$ follow (respectively) a chi-squared distribution with $1$ df and a chi-squared distribution with $n-1$ df, and that they are independent.

So, in my mind, the ratio $\frac{\frac{SSR}{\sigma^2} / 1}{\frac{SST}{\sigma^2} / (n-1)}$ should follow $F(1, n-1)$, a Fisher-Snedecor distribution with $df$ $1$ and $n-1$...

Instead of finding this result, what I see written everywhere is that $(n-2) \frac{R^2}{1-R^2}$ follows $F(1, n-2)$...

Why do people use this latter form instead of the former ? Are they equivalent ? Something escapes me…

$\endgroup$
3
$\begingroup$

In a model with a single explanatory variable and an intercept term you have regression degrees-of-freedom $DF_R = 1$ and residual degrees-of-freedom $DF_E = n-2$, so you get:

$$\begin{equation} \begin{aligned} F \equiv \frac{MSR}{MSE} &= \frac{SSR / DF_R}{SSE / DF_E} \\[6pt] &= \frac{DF_E}{DF_R} \cdot \frac{SSR}{SSE} \\[6pt] &= \frac{DF_E}{DF_R} \cdot \frac{SSR}{SST-SSR} \\[6pt] &= (n-2) \cdot \frac{R^2}{1-R^2}. \\[6pt] \end{aligned} \end{equation}$$

In your own calculations, you appear to have used an incorrect value for the residual degrees-of-freedom ($n-1$ instead of $n-2$) so your result is not equivalent to the correct form.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.