I had a few true / false questions on a practice test that I would like to discuss if possible.

  • A value $R^2$ close to 1 indicates the linear regression is a good fit to data

Yes, but I am not sure how to reason it other than intuitively. More precisely, why do we decide to define $R^2$ as 1-(sum of residuals/total sum of squares)?

  • The estimate of the error variance $s^2$ is a random variable

I believe it is because how the error varies is normally distributed if I remember correctly. If anyone could elaborate it would be much appreciated.

  • $∑(Y_{1,i}−Y_{2,i})^2 = 0$, where $Y_1$ is the predicted value and $Y_2$ is the actual value.

Since the sum of residuals is zero and since this is equivalent to the sum of residuals it must be zero.

  • $\begingroup$ Welcome to the site, @user26091. I took the liberty of editing your question with $\LaTeX$. Please make sure it still says what you want it to. $\endgroup$ – gung - Reinstate Monica May 25 '13 at 1:11
  • $\begingroup$ Thanks! It does not quite say what I want it to however but close. $\endgroup$ – user26091 May 25 '13 at 1:36
  • $\begingroup$ I thought there was something wrong with c, but I couldn't figure out what it was supposed to be. $\endgroup$ – gung - Reinstate Monica May 25 '13 at 1:39
  • $\begingroup$ Since it's for the purpose of study, please add the self-study tag. I took a stab at improving "c" - if that's wrong, please clarify. $\endgroup$ – Glen_b -Reinstate Monica May 25 '13 at 1:45
  • 2
    $\begingroup$ @PeterFlom, $\LaTeX$ is "\$\LaTeX\$". $\endgroup$ – gung - Reinstate Monica May 25 '13 at 13:41

c) Is definitely wrong, because it is the sum of SQUARED residuals. It would be the variance of prediction errors, if you divided it by the degrees of freedom, assuming that the mean of prediction errors is 0.

a) $R^2$ is the proportion of explained variation on the whole variation. The formula is just a different representation of $R^2$.

b) I would say, everything calculated from random variables is a random variable itself.

  • $\begingroup$ Note that (c) is not divided by df. $\endgroup$ – gung - Reinstate Monica May 25 '13 at 14:36
  • $\begingroup$ Right, so if it was NOT the square it would be zero. $\endgroup$ – user26091 May 26 '13 at 18:51
  • $\begingroup$ Why is anything calculated from random variables necessarily a random variable? Is this a statistical version of function composition in some ways? $\endgroup$ – user26091 May 26 '13 at 18:53
  • $\begingroup$ Intuitively I would say, calculating a parameter using a sample from a random variable, the randomness flows into your parameter. If you draw another sample from the same variable, you would most likely get a slightly different estimate, from a new sample another one, and so on... $\endgroup$ – DatamineR May 26 '13 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.