3
$\begingroup$

On a business task, I perform simple linear regression with an independent variable $X$ and a dependent variable $Y$. The result is a linear function with slope $β_1$. If $R^2$ (coefficient of determination) = 1, I conclude that an increase of 1 in $X$ will result in an increase of $β_1$ in $Y$. If $R^2$ = 0, I conclude that an increase of 1 in $X$ will on average result in an increase of 0 in $Y$.

  1. Are my conclusions correct?
  2. If $R^2 = 0.5$, can I conclude that an increase of 1 in $X$ will on average result in an increase of $0.5β_1$ in $Y$?
  3. If 2. is incorrect, is there a way to infer such a relationship from a linear regression model («an increase of 1 in $X$ will on average result in an increase of ... in $Y$»)?
$\endgroup$
3
$\begingroup$

Firstly lets remember $R^2$==1-$SS_{reg}$/$SS_{tot}$. Where $SS_{reg}$ is the sum of squared errors from your model:

$Y=\beta_{1}X+\epsilon$ (Where $\epsilon$ is the error in your prediction)

And $SS_{tot}$ is the sum of squared errors when you assume $Y$ = $\bar{Y}$. This is the same as the linear model $Y=\beta_{0}+\epsilon$, otherwise known as the NULL model.

1.Your first conclusions are correct. Because $R^2$ shows how well movements in $Y$ are explained by your model $\beta_{1}X+\epsilon$. An $R^2$ of 1 means your model perfectly explains movements of $Y$ that is $SS_{reg}$=0 i.e. $\epsilon=0$. So you can conclude an increase in $X$ by 1 will definitely result in $Y$ increasing by $\beta_{1}$. Likewise an $R^2$ of 0 means your model $\beta_{1}X+\epsilon$ does no better in explaining movements in $Y$ than just assuming $Y$ will be the average $\bar{Y}$. (That is $SS_{reg}$=$SS_{tot}$). i.e. $X$ has no bearing on $Y$.

2.If $R^2$=0.5 it means your model explains 50% of the movement in $Y$ when compared to simply stating $Y$ = $\bar{Y}$. The unexplained movement in $Y$ is captured by your error term $\epsilon$, which is white noise. This means when $X$ increases by 1 $Y$ on average will increase by $\beta_{1}$ but due to the unexplained white noise you might not see this in in the data i.e. in every instance of $Y_{i}$.

3.Even where $R^2$<1 your interpretation of $\beta_{1}$ is still the average increase in $Y$ when $X$ increases by 1. It just means there is noise that has been unaccounted for in the model. Adding other predictors into the model may well change the value of $\beta_{1}$, but given the model $\beta_{1}X+\epsilon$, $\beta_{1}$ should be interpreted as the average increase in $Y$ when $X$ increases by 1. The only time this is inappropriate is when $R^2$==0 because in that case $X$ has no relationship with $Y$.

$\endgroup$
  • $\begingroup$ In this business case, the individual results are not important, only the long-term average. Does this mean that I from a business point-of-view can ignore $R^2$ as long as I can show with a significance test that $R^2$ > 0 ? $\endgroup$ – matthiash Mar 16 '17 at 11:55
  • $\begingroup$ $R^2$ is useful in telling you whether your model $\hat{\beta}_{1}X_{1}$ is a good predictor of $Y$ or not so I wouldn't ignore it but it doesn't change the interpretation of $\beta_{1}$, assuming $\hat{\beta}_{1}$ is statistically significant. You may want to add other predictors to your model that way you can say when holding for $X_{2}$, $X_{3}$, etc $Y$ increases on average by $\beta_{1}$ when $X_{1}$ increases by 1. At the moment your model isn't holding for any other predictors so $\hat{\beta}_{1}$ may not be a good estimation of the population $\beta_{1}$ $\endgroup$ – Morgan Ball Mar 16 '17 at 12:02
  • $\begingroup$ To be clear I've used $\hat{\beta}_{1}$ in the above comment to differentiate between your estimated coefficient and the actual population coefficient $\beta_{1}$ you're trying to discover. In my answer any reference to $\beta_{1}$ is a reference to the estimated relationship between $X$ and $Y$. In reality because your not holding for other variables $\hat{\beta}_{1}$ may be a biased estimator of the population $\beta_{1}$ $\endgroup$ – Morgan Ball Mar 16 '17 at 12:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.