2
votes
When is least squares better than reduced major axis?
To start with, the major theoretical advantage of RMA estimation is that it is symmetric with respect to the inputs --- i.e., if you exchange the response variable in the model with an explanatory ...
- 94.4k
2
votes
Why OLS perform better than LASSO?
This is one example of a "curse of dimensionality" described for LASSO in An Introduction to Statistical Learning, second edition, at the end of Chapter 6.
With a large number of potential ...
- 62.2k
2
votes
Accepted
Interpretation problems of linear model with no predictors
The linear regression model is
$$
Y_i = \beta_0 + \beta_1 X_1 + \dots + \beta_k X_k + \varepsilon_i
$$
with $\varepsilon_i$ being i.i.d. Gaussian noise with mean equal to zero. The model estimates the ...
Tim♦
- 113k
2
votes
Accepted
How to interpret coefficients vs relative importance of variables in linear regression?
As you note, model coefficients in a linear regression will tell you about change in predicted outcome for a one unit change in the predictor. Coefficients, standardized or not, then relate mostly to ...
- 601
2
votes
Equality-constrained least-squares when the matrix is singular
If it is possible to obtain a unique solution to $Ax=b$ subject to the constraints $Cx =d,$ then you can obtain it with Lagrange multipliers.
To be explicit, suppose $A$ is an $n\times p$ matrix, $x$ ...
- 287k
2
votes
Accepted
Does an endogenous variable bias the coefficient of the exogenous one?
Well, except in the multivariate normal case, zero covariance does not imply independence. You have not specified any distributions, so we cannot assume multivariate normal distributions. So ...
- 4,225
2
votes
Accepted
Inclusion of year and seasons as variable for regression with non-stationary response?
Regarding OLS only makes sense if both the response and explanatory variables are non-stationary, actually, it is the opposite: non-stationary should be replaced by stationary. Though as Chris Haug ...
- 55.4k
1
vote
Is it possible to derive the joint probability distribution of squared OLS residuals under the classical linear regression assumptions?
I've managed to come up with an answer to my question as follows.
Bivariate Case
Let ${(U_1,V_1),\ldots,(U_k,V_k)}$ be an independent random sample of size $k$ from a bivariate normal distribution ...
- 161
1
vote
OLS estimator question: using a subset versus using a dummy-interacted variables
The way that you structured df4 you effectively only included interaction (product) terms between the binary x4 and the ...
- 62.2k
1
vote
What is the intercept in a regression model with demeaned dependent variable?
The intercept is the predicted response when all the predictors are zero, a point that may or not may not occur within the range of the dataset.
Shifting the response so that it has mean zero doesn't ...
- 48.9k
1
vote
Linear Least Squares vs Ordinary Least Squares
No difference at all. These are equivalent.
1
vote
Accepted
Linear regression has good performance in validation set despite not meeting the linearity assumption
The idea that you'd need to "make the data meet" certain model assumptions is wrong, as model assumptions are never perfectly fulfilled anyway. In particular, formal model assumptions almost ...
- 11.4k
Only top scored, non community-wiki answers of a minimum length are eligible