# Tag Info

### When is least squares better than reduced major axis?

To start with, the major theoretical advantage of RMA estimation is that it is symmetric with respect to the inputs --- i.e., if you exchange the response variable in the model with an explanatory ...

### Why OLS perform better than LASSO?

This is one example of a "curse of dimensionality" described for LASSO in An Introduction to Statistical Learning, second edition, at the end of Chapter 6. With a large number of potential ...
Accepted

### Interpretation problems of linear model with no predictors

The linear regression model is $$Y_i = \beta_0 + \beta_1 X_1 + \dots + \beta_k X_k + \varepsilon_i$$ with $\varepsilon_i$ being i.i.d. Gaussian noise with mean equal to zero. The model estimates the ...
Accepted

### How to interpret coefficients vs relative importance of variables in linear regression?

As you note, model coefficients in a linear regression will tell you about change in predicted outcome for a one unit change in the predictor. Coefficients, standardized or not, then relate mostly to ...

### Equality-constrained least-squares when the matrix is singular

If it is possible to obtain a unique solution to $Ax=b$ subject to the constraints $Cx =d,$ then you can obtain it with Lagrange multipliers. To be explicit, suppose $A$ is an $n\times p$ matrix, $x$ ...
Accepted

### Does an endogenous variable bias the coefficient of the exogenous one?

Well, except in the multivariate normal case, zero covariance does not imply independence. You have not specified any distributions, so we cannot assume multivariate normal distributions. So ...
Accepted

### Inclusion of year and seasons as variable for regression with non-stationary response?

Regarding OLS only makes sense if both the response and explanatory variables are non-stationary, actually, it is the opposite: non-stationary should be replaced by stationary. Though as Chris Haug ...
1 vote

### Is it possible to derive the joint probability distribution of squared OLS residuals under the classical linear regression assumptions?

I've managed to come up with an answer to my question as follows. Bivariate Case Let ${(U_1,V_1),\ldots,(U_k,V_k)}$ be an independent random sample of size $k$ from a bivariate normal distribution ...
1 vote

### OLS estimator question: using a subset versus using a dummy-interacted variables

The way that you structured df4 you effectively only included interaction (product) terms between the binary x4 and the ...
1 vote

### What is the intercept in a regression model with demeaned dependent variable?

The intercept is the predicted response when all the predictors are zero, a point that may or not may not occur within the range of the dataset. Shifting the response so that it has mean zero doesn't ...
1 vote

### Linear Least Squares vs Ordinary Least Squares

No difference at all. These are equivalent.
1 vote
Accepted

### Linear regression has good performance in validation set despite not meeting the linearity assumption

The idea that you'd need to "make the data meet" certain model assumptions is wrong, as model assumptions are never perfectly fulfilled anyway. In particular, formal model assumptions almost ...

Only top scored, non community-wiki answers of a minimum length are eligible