6
$\begingroup$

$R^2 = \frac{SSREG}{SSTOT}$ or $R^2 = 1-\frac{SSRES}{SSTOT}$

If $X$ is the predictor random variable for science SAT

and $Y$ is the predictor random variable science GPA

given by equation $$\hat Y = -1 + 0.006X$$

and $R^2 = 0.3$

what do we say the relationship between $X$ and $Y$ is in this case given the coefficient of determination?

$\endgroup$
2
  • $\begingroup$ In your question the R-squared value is 3.0, is this correct? I ask because my understanding is that R-squared should be between 0.0 and 1.0. $\endgroup$ Commented May 23, 2019 at 1:25
  • $\begingroup$ 0.3 I meant, sorry. $\endgroup$
    – Tree Garen
    Commented May 23, 2019 at 4:28

2 Answers 2

3
$\begingroup$

$$R^2 = 1-\frac{SSRES}{SSTOT}$$

When there there is no residual variation ($SSRES=0$) then the regression line fits the data perfectly and $R^2$ is 1, whereas when there is there is large residual variation, $\frac{SSRES}{SSTOT}$ approaches 1 and so $R^2$ approaches 0.

what do we say the relationship between X and Y is in this case given the coefficient of determination?

In this case (simple linear regression) $R$ is the Pearson correlation coefficient between the two variables, and the standardized regression coefficient, whereas $R^2$ measures the proportion of variability explained by the model. Since your regression equation has a coefficient for $X$ of 0.006 and $R^2$ is 0.3 it is obvious that the two variables are measured on different scales. If you standardized the two variables, $R$ and the coefficient for $X$ would be the same. See here for further details.

So the interpretation is that there is a positive relationship between X and Y, and that for every increase of 1 unit in $X$ the model predicts a 0.006 unit increase in $Y$, and that the model explains 30% of the variability in $Y$.

It would be a good idea to plot your data and consider the assumptions of the model (especially the assumption of linearity), and if you intend to use the model for inference then further checks should be made.

$\endgroup$
1
$\begingroup$

I think it's important to consider what regression is doing. Then the coefficient of determination makes sense.

Let's say that we collect some data on the heights of people. From our data, we find a mean of 5'2" with a middle range (Q1 to Q3) of 4'2" to 6'2". Given a new person, what height do you guess?

Depending on your application, mean may or may not be what you're looking for, but let's say it is, since OLS regression is predicting conditional means. You could guess 5'2" with a range of 4'2" to 6'2", but that's such a wide range! You kind of have no idea how tall this random subject will be. However, in the absence of any other knowledge, you know that the mean value gets the right answer on average, so you guess the mean. You get the answer wrong--by a lot.

However, you could have collected other information about the people whose heights you measured. I would expect height differences between men and women. Certainly I expect height differences at different ages.

Now you know that the person whose height you have to guess is a 40-year-old man. You go look at your subjects with known height and don't have any 40-year-old men, but you have men who are 39, 41, and 42, who average 5'9" with a range of 5'8" to 5'10". You have much more confidence in your answer of 5'9". In fact, the subject was 5'8". By doing regression on the age and gender, you have decreased your error from 6" to 1".

The goal of regression is to decrease that variability by using other information. That other information form the predictive variables in your model. Regression then predicts the mean of a conditional distribution, conditioned on that other information (e.g. male and 40 years old).

Let's get back to the formula for the coefficient of determination. The total sum of squares is by how much you miss the correct value by guessing the average of all observations. The sum of squared errors is by how much you miss the right answer by predicting the mean of the conditional distribution. The sum squared of the regression is by how much you decrease your error by accounting for the additional information. Instead of naively guessing the overall mean, you something about the subject and tighten up your guess.

When we say that the coefficient of determination is SSReg/SSTotal, we are saying what percentage of the variability in the observations is accounted for by the additional information (predictive variables). If we have a conditional distribution with little variability, then we can make a very tight guess about what a new observation would be.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.