Linked Questions
22 questions linked to/from Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$
1
vote
1
answer
150
views
Making the mindset transition - projection in OLS regression [duplicate]
For example, the space spanned by the columns in
$$\mathbf{X} = \begin{bmatrix}
0 & 0 \\
1 & 0 \\
0 & 1
\end{bmatrix}$$
is the y-z plane.
Further,
$$\mathbf{X'X} = \begin{bmatrix}
1 &...
- 11
220
votes
5
answers
279k
views
How exactly does one “control for other variables”?
Here is the article that motivated this question: Does impatience make us fat?
I liked this article, and it nicely demonstrates the concept of “controlling for other variables” (IQ, career, income, ...
- 3,017
154
votes
3
answers
90k
views
Removal of statistically significant intercept term increases $R^2$ in linear model
In a simple linear model with a single explanatory variable,
$\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$
I find that removing the intercept term improves the fit greatly (value of $R^2$ ...
- 2,392
34
votes
3
answers
65k
views
How can the regression error term ever be correlated with the explanatory variables?
The first sentence of this wiki page claims that
In econometrics, an endogeneity problem occurs when an explanatory variable is correlated with the error term.
My question is that how can this ever ...
- 1,046
32
votes
6
answers
58k
views
Why is $SST=SSE + SSR$? (One variable linear regression)
Note: $SST$ = Sum of Squares Total, $SSE$ = Sum of Squared Errors, and $SSR$ = Regression Sum of Squares. The equation in the title is often written as:
$$\sum_{i=1}^n (y_i-\bar y)^2=\sum_{i=1}^n (...
- 421
26
votes
6
answers
19k
views
Reason for not shrinking the bias (intercept) term in regression
For a linear model $y=\beta_0+x\beta+\varepsilon$, the shrinkage term is always $P(\beta) $.
What is the reason that we do not shrink the bias (intercept) term $\beta_0$? Should we shrink the bias ...
- 855
35
votes
2
answers
17k
views
What is the distribution of $R^2$ in linear regression under the null hypothesis? Why is its mode not at zero when $k>3$?
What is the distribution of the coefficient of determination, or R squared, $R^2$, in linear univariate multiple regression under the null hypothesis $H_0:\beta=0$?
How does it depend on the number ...
- 107k
15
votes
3
answers
12k
views
What is an example of perfect multicollinearity?
What is an example of perfect collinearity in terms of the design matrix $X$?
I would like an example where $\hat \beta = (X'X)^{-1}X'Y$ can't be estimated because $(X'X)$ is not invertible.
- 347
41
votes
1
answer
28k
views
Is there any difference between $r^2$ and $R^2$?
The correlation coefficient is usually written with a capital $R$ but sometimes not. I wonder if there really is a difference between $r^2$ and $R^2$? Can $r$ mean something else than a correlation ...
- 627
12
votes
3
answers
93k
views
Do correlation or coefficient of determination relate to the percentage of values that fall along a regression line?
Correlation, $r$, is a measure of linear association between two variables. Coefficient of determination, $r^2$, is a measure of how much of the variability in one variable can be "explained by" ...
- 340
17
votes
3
answers
6k
views
The equivalence of sample correlation and R statistic for simple linear regression
It is often stated that the square of the sample correlation $r^2$ is equivalent to the $R^2$ coefficient of determination for simple linear regression. I have been unable to demonstrate this myself ...
- 313
20
votes
2
answers
7k
views
Why squaring $R$ gives explained variance?
This may be a basic question, but I was wondering why an $R$ value in a regression model can simply be squared to give a figure of explained variance?
I understand that $R$ coefficient can give the ...
- 199
12
votes
1
answer
9k
views
Data space, variable space, observation space, model space (e.g. in linear regression)
Suppose we have the data matrix $\mathbf{X}$, which is $n$-by-$p$, and the label vector $Y$, which is $n$-by-one. Here, each row of the matrix is an observation, and each column corresponds to a ...
- 1,122
7
votes
3
answers
14k
views
Why does the sum of residuals equal 0 from a graphical perspective?
I've seen the proof for why in least squares regression the sum of residuals is always equal to 0, and I kind of understand why from that algebraic perspective. Basically, you're finding the minimum ...
- 207
10
votes
2
answers
2k
views
Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?
In linear regression I have come across a delightful result that if we fit the model
$$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$
then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data,
$$R^...
- 5,415