1
$\begingroup$

I find this question about linear regression, "I have a n sample $(X,Y)_1,...,(X,Y)_n$, I do a linear regression based on those data (I've guessed that the predictor is X), and I calculate the $R^2$.

Now I created a 2n sample based on those data, my sample is $(X,Y)_1,... ,(X,Y)_n, ((X,Y)_{n+1}=(X,Y)_1),...((X,Y)_{2n}=(X,Y)_n)$. Basically, in order to build this 2n sample, I use the previous data that I have. Now the question is how the $R^2$ have changed if I do a linear regression ?

I would say that the $R^2$ doesn't change, using the formula $R^2= \frac{\text{explained variance}}{\text{total variance}}$, As we are just adding the same data. If we add regressors I know that the $R^2$ increases (overfitting that why we should measure of the goodness of the fit such that the adjusted $R^2$, or the AIC criterion)

Can you tell me if I have missed something ? I hope I have made myself clear

$\endgroup$

1 Answer 1

3
$\begingroup$

It doesn't change. Take in account that the determination coefficient $R^2$ is the square of the linear correlation coefficient $r$. Thus, if the correlation coefficient doesn't change, then $R^2$ won't change. Now the proof that $r$ won't change:

Given that $r$ is the correlation coefficient of the first sample and $r'$ the correlation coefficient of the second sample, we have to prove that $r=r'$. We have the following: $$ r'=\frac{\sum_{i=1}^{2n}{(x_i-\bar{X})(y_i-\bar{Y})}}{2n} $$ But because we just repeat the first half of the data, then: $$ r'=\frac{2\sum_{i=1}^{n}{(x_i-\bar{X})(y_i-\bar{Y})}}{2n}=\frac{\sum_{i=1}^{n}{(x_i-\bar{X})(y_i-\bar{Y})}}{n}=r $$

Proved!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.