I was reading a document about the coefficient of determination and saw that $R^2$ shows how much of the variation in y is represented by the variation in x. What I couldn't understand was the part variation in x. Because we don't calculate variance or anything, we simply write (y_actual - (mx+b))^2 for each point. Can someone explain to me what the variation in x refers to?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
The estimated coefficients $\hat m$ and $\hat b$ are being calculated using statistics of $x$, and they describe the how $x$ is related to $y$ (linearly), which means variations in $y$ are explained by variations in $x$. If the coefficient of determination is $1$, $x$ can perfectly explain $y$. For example, if $m=0$ and $b=\bar y$, your regression model will always predict the mean, and the variation in $y$ is not explained at all ($r^2=0$).
-
$\begingroup$ I know that. What I meant is that is it different than the variance of x values? because if variation and variance the same, wouldn't that mean that y is affected by the variance, a measure of spread, of x values. $\endgroup$ Commented May 11, 2021 at 16:19
-
$\begingroup$ Not the same always. Variation is a more general term w/o explicit formula, and can be quantified with variance (then they're equal). $\endgroup$– gunesCommented May 15, 2021 at 22:08
$\begingroup$
$\endgroup$
The way that I think of regression is that we want to make accurate predictions of some variable of interest ($Y$). If we just have measurements of that variable, all we can use is that variable, and we cannot explain its variability. If we measure some determinants ($X$) of that variable, however, then it might be that some of the variability in $Y$ can be explained by the fact that $X$ is not constant. That is, some of the reason that $Y$ varies is because $X$ varies.
As an example, consider predicting the height of a human. If all you know is that the subject is a human, all you have to go by is the overall distribution of human heights, and you cannot explain any of the variability in human heights. However, we know some determinants of human height. Age is a big one: as humans grow up, they get taller. That is, part of the reason why there is such variation in human height is because a major determinant of human height, age, has variability. Of course human heights will vary if this age determinant of human height varies!
When you consider "variance" as the mathematical measure of the colloquial term "variation", this leads to using a phrase like "the proportion of variance in our variable of interest that is explained by the variance in some observed determinant(s) of that variable of interest."
This is enough for me. If you want to get into the math, perhaps you can argue that $R^2$ is related to the squared correlation between the outcome $y$ and predictor $x$, so the denominator of that correlation contains the standard deviation of $x$ that gets squared to the variance of $x$. Another thought could be to use the fact that $\left(y - (mx + b)\right)^2$ can be expanded to involve a square of $x$, which would be related to the variance of $x$.
