# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

2,512 questions
Filter by
Sorted by
Tagged with
4 views

### implement i.year i.id FE from stata in python [closed]

When I have to control for fixed effects (time and id) in Stata, I usually run the regression with i.year and i.id (these are my var names). Now, I have to use python for a different project because ...
12 views

45 views

### When is least squares better than reduced major axis?

Consider two linear regression methods: least squares regression (LSR) reduced major axis (RMA) I know the definitions of both regression methods but I would like to know when is the LSR better than ...
• 31
1 vote
16 views

### Comparing Activities with regression

I am trying to find out what is the effect of activities(like jumping, weight lifting etc.) on behavior (such as attitude towards participating in a marathon). (sample size of 60 observations for each ...
• 11
21 views

### What happens if the "coefficients" in the data generating process are correlated with the variance of the error term?

Suppose we are interested in estimating a regression of the form $$y = \beta x + \epsilon$$ but in the data generating process, $\beta$ is decreasing in $\mathbb{E}[\epsilon^2]$. For example, there ...
1 vote
46 views

### Why OLS perform better than LASSO?

I am comparing OLS and LASSO regression for survey data. I have n>p, but I think my data is high-dimensional data as the p is 3000 and n is 48000. I am using k cross-validation. The results are ...
• 11
13 views

### Can introducing time fixed effects variable into a PanelOLS decrease overall and between R^2?

I am trying to find if there is a relationship between the number of people employed by the tech industry within a city and wages in that city. I ran two Linear Regressions on my data. The first one ...
7 views

### Distinguishing between effects of two variables on y

Assume that we have the linear regression model: $$y=\beta+\beta_{1}x+\epsilon$$ We estimate the model by OLS, and we get $\hat{\beta}_{1},$ However, there is another variable $z,$ which is both ...
26 views

### For all datasets with a binary outcome, will linear regression always yield betas with a smaller standard error compared to logistic regression?

Any cases where the betas' standard errors from logistic regression will be smaller than linear regression, after converting from log odds space to probability space?
9 views

### Is my survey data structured correctly to run OLS regression in R?

I have data from two surveys that were launched at the same time and am trying to find whether there is a statistically significant result on the number of respondents as a consequence of various ...
9 views

### GPD and GEV Fitting: Maximum Likelihood vs. Least Squares

I am trying to build a model based on real world data which involves fitting generalized extreme value distributions and generalized Pareto distributions. Most literature immediately turns to the ...
1 vote
40 views

### Baseline variable in regression

I am currently looking at this paper: https://www.nature.com/articles/s41591-021-01487-3 The equation for (1) includes a variable for a baseline value. I am confused as to why they do this as I ...
• 31
23 views

### How do I derive the variance of OLS estimators when I have dummy explanatory variables?

I know this isn't the smartest question, however I need to derive the variance of the OLS estimators in a Simple Linear Regression Model when the explanatory variable is a dummy one and all the ...
7 views

### Using IV when regressor is not endogenous

Suppose I have a single regressor model and the regressor itself is uncorrelated with the error term. If I were to use IV estimation to estimate the coefficient, would the estimate be incorrect, and ...
26 views

27 views

### Unbiasedness and consistency of OLS in an AR(1) model with AR(1) residuals [duplicate]

consider equation 1 : , Now let , where the error component is iid with mean 0 and constant variance, and Is the OLS estimator of the coefficients in equation 1 unbiased and consistent under this ...
1 vote
37 views

### Different number of observations after including control variables

I have two regression models. I am using paneled data on individuals from 2010 up to 2019. For some individuals, I have several years of observations, whereas for others, there are only 2 or so. The ...
• 41
1 vote
48 views

### Do I need to transform/standardise my dependent variable?

Attached are the results and the residual plot for my regression of control variables on CEO compensation (TDC1). When I look at the plot my main concerns are the outliers (which I checked to be ...
1 vote
26 views

### Gradient of the second order term of Newton's Method

I know that Netwon's method can be pushed to the second order using the 1st Taylor expansion. However, how can I generalize Netwon's method to take x_0 as a vector and have the ability take the ...
1 vote
10 views

### Parametric bootstrap *prediction* interval with heteroskedasticity and sandwich parameter covariance matrix

The sandwich estimator for OLS regressions where heteroskedasticity is suspected is $$var(\hat\beta) = (X'X)^{-1}X'ee'X(X'X)^{-1}$$ If I want confidence intervals on predictions, I can just take ...
• 12.1k
1 vote
11 views

### How to derive least squares estimators from normal equations with work shown

My last question was closed, so I made a new one that is more relevant and has my full calculations thus far. I have a scenario where I have been given some information, such as normal equations from ...
• 65
1 vote
88 views

### How to obtain least squares when $X^TX$ cannot be inverted

This work is all theoretical and for school, so we were only provided this information to work with, no actual y values. I have a simple linear model I have been asked to translate into a matrix, ...
• 65