2 votes

When is least squares better than reduced major axis?

To start with, the major theoretical advantage of RMA estimation is that it is symmetric with respect to the inputs --- i.e., if you exchange the response variable in the model with an explanatory ...
user avatar
  • 94.4k
2 votes

Why OLS perform better than LASSO?

This is one example of a "curse of dimensionality" described for LASSO in An Introduction to Statistical Learning, second edition, at the end of Chapter 6. With a large number of potential ...
user avatar
  • 62.2k
2 votes
Accepted

Interpretation problems of linear model with no predictors

The linear regression model is $$ Y_i = \beta_0 + \beta_1 X_1 + \dots + \beta_k X_k + \varepsilon_i $$ with $\varepsilon_i$ being i.i.d. Gaussian noise with mean equal to zero. The model estimates the ...
user avatar
  • 113k
2 votes
Accepted

How to interpret coefficients vs relative importance of variables in linear regression?

As you note, model coefficients in a linear regression will tell you about change in predicted outcome for a one unit change in the predictor. Coefficients, standardized or not, then relate mostly to ...
user avatar
  • 601
2 votes

Equality-constrained least-squares when the matrix is singular

If it is possible to obtain a unique solution to $Ax=b$ subject to the constraints $Cx =d,$ then you can obtain it with Lagrange multipliers. To be explicit, suppose $A$ is an $n\times p$ matrix, $x$ ...
user avatar
  • 287k
2 votes
Accepted

Does an endogenous variable bias the coefficient of the exogenous one?

Well, except in the multivariate normal case, zero covariance does not imply independence. You have not specified any distributions, so we cannot assume multivariate normal distributions. So ...
user avatar
2 votes
Accepted

Inclusion of year and seasons as variable for regression with non-stationary response?

Regarding OLS only makes sense if both the response and explanatory variables are non-stationary, actually, it is the opposite: non-stationary should be replaced by stationary. Though as Chris Haug ...
user avatar
1 vote

Is it possible to derive the joint probability distribution of squared OLS residuals under the classical linear regression assumptions?

I've managed to come up with an answer to my question as follows. Bivariate Case Let ${(U_1,V_1),\ldots,(U_k,V_k)}$ be an independent random sample of size $k$ from a bivariate normal distribution ...
user avatar
1 vote

OLS estimator question: using a subset versus using a dummy-interacted variables

The way that you structured df4 you effectively only included interaction (product) terms between the binary x4 and the ...
user avatar
  • 62.2k
1 vote

What is the intercept in a regression model with demeaned dependent variable?

The intercept is the predicted response when all the predictors are zero, a point that may or not may not occur within the range of the dataset. Shifting the response so that it has mean zero doesn't ...
user avatar
  • 48.9k
1 vote

Linear Least Squares vs Ordinary Least Squares

No difference at all. These are equivalent.
user avatar
1 vote
Accepted

Linear regression has good performance in validation set despite not meeting the linearity assumption

The idea that you'd need to "make the data meet" certain model assumptions is wrong, as model assumptions are never perfectly fulfilled anyway. In particular, formal model assumptions almost ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible