Linked Questions

1
vote
1answer
6k views

How do you calculate the Ordinary Least Squares estimated coefficients in a Multiple Regression Model? [duplicate]

I recently began learning about OLS estimation of multiple regression models and came across the following formulas explaining the calculations: What would the formulas be for an OLS regression model ...
1
vote
0answers
1k views

Confusion regarding “regression by successive orthogonalization” [duplicate]

In trying to answer a question here on Cross Validated, I was re-reading Section 3.2.3, specifically Algorithm 3.1 from Elements of Statistical Learning. What I followed from this is that, given a ...
3
votes
0answers
617 views

Least squares - why multiply both sides by the transpose? [duplicate]

The formula: $A^T(b - Ax) = 0\\ A^Tb = A^TAx\\ x = (A^TA)^{-1}A^Tb$ What is the reason for multiplying both sides by the transpose of A?
0
votes
0answers
328 views

Matrix of regression coefficients [duplicate]

Right above this link here it's mentioned that the matrix of regression coefficients, $(\sum_{XX})^{-1}\sum_{XY}$, corresponds to the coefficients of an OLS solution. But I can't really see it for ...
0
votes
1answer
160 views

How to derive the formula for coefficient (slope) of a simple linear regression line? [duplicate]

There is a formula for calculating slope (Regression coefficient), b1, for the following regression line: y= b0 + b1 xi + ei (alternatively y'(predicted)=b0 + b1 * x); which is b1=(∑(xi-Ẋ) * (yi-...
0
votes
1answer
69 views

Formula for B values in Multiple Regression [duplicate]

I am curious what the formula for B values is when you have more than two variables. While I know any software can easily tell me the answer, I do not want to use it to figure out the B values, I want ...
1
vote
0answers
24 views

Equation (3.27) from Elements of statistical Learning [duplicate]

It seems the equation below has a typo? Shouldn't $\boldsymbol{y}$ be $\boldsymbol{y} - \boldsymbol{1}\bar{y}$?
1
vote
0answers
20 views

Obtaining the $j$th component of the OLS - an explanation [duplicate]

In a linear regression setting, I've seen that the $j$th component of the ordinary least square estimator $\hat{\beta}_j$ can be obtained as follows: $$\hat{\beta}_j = Y^T Z^{(j)} / (X^{(j)})^T Z^{(j)}...
169
votes
5answers
215k views

How exactly does one “control for other variables”?

Here is the article that motivated this question: Does impatience make us fat? I liked this article, and it nicely demonstrates the concept of “controlling for other variables” (IQ, career, income, ...
47
votes
6answers
29k views

What algorithm is used in linear regression?

I usually hear about "ordinary least squares". Is that the most widely used algorithm used for linear regression? Are there reasons to use a different one?
17
votes
1answer
9k views

Question on how to normalize regression coefficient

Not sure if normalize is the correct word to use here, but I will try my best to illustrate what I am trying to ask. The estimator used here is least squares. Suppose you have $y=\beta_0+\beta_1x_1$, ...
10
votes
3answers
2k views

Intuition behind $(X^TX)^{-1}$ in closed form of w in Linear Regression

The closed form of w in Linear regression can be written as $\hat{w}=(X^TX)^{-1}X^Ty$ How can we intuitively explain the role of $(X^TX)^{-1}$ in this equation?
7
votes
3answers
409 views

Converting the beta coefficient from matrix to scalar notation in OLS regression

I've found for my econometrics exams that if I forget the scalar notation, I can often save myself by remembering the matrix notation and working backwards. However, the following confused me. Given ...
3
votes
1answer
2k views

How to proof relationship between inverse covariance matrix and linear regression coefficients?

Edited: I would like to work out the above relationship, more precisely: Let $(Y_{1}, ..., Y_{m})$ be a zero-mean vector with covariance matrix $\Sigma$, and let $S \subset \{1, ..., m\}.$ The ...
2
votes
1answer
980 views

OLS estimator for regression without intercept [duplicate]

Consider a linear regression model: $Y_i = \beta_1 A_i + \beta_2 B_i + u_i$ where all variables are assumed to have mean 0, and $A_{i}$ is distributed independently of both $B_{i}$ and $u_{i}$, but $...

15 30 50 per page