# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

1,629 questions
48 views

### Is constrained (nonnegative) least squares a form a regularisation?

Is nonnegative least squares already a form of regularisation? By adding a constraint that $\beta \geq 0$ (the coefficients), does it make sense to add another regularisation term as in LASSO or ridge ...
71 views

### What is the difference between $E[\varepsilon\mid X]=0$ and $E[\varepsilon X]=0$ in OLS regression?

Why is the assumption $E[\varepsilon X]=0$ weaker than $E[\varepsilon\mid X]=0$?
145 views

47 views

### In OLS, while using log-log and linear-log transforamtions, is valid to transform some regressors only?

In OLS I was wondering if it is valid to log-transform some regressors only. Specifically, continuous regressors, because it is advised not to transform binary or categorical variables. For instance, ...
183 views

### VIF Drops Significantly When I Delete Some Dummy Variables

Is my model valid even with the high VIF? Does it matter which dummy variable I drop as the reference point? I have a a category variable (Fruit) that I converted ...
70 views

10 views

### Determining the Relationship Between Monte Carlo Breaks and Model Volatility

I'm looking for a statistical test to understand the relationship (if any) between the model volatilities of a stochastic process, and the occurrence of a'break', defined as an instance when an ...
78 views

41 views

### Intuitive explanation from regression coefficient estimate formula

Can someone provide an intuitive explanation of why the OLS regression estimate, of y=a+bx, b have the form b=cov(x,y)/V(x). How intuitively are the covariance and variance related in this?
36 views

### Assumptions of linear fit; linearity and homoscedasticity

I'm reading about the assumptions of taking a linear fit between two variables from here, and that source says: For diagnosing non-linearity: nonlinearity is usually most evident in a plot of ...
40 views

### Unbiasedness and Consistency of DID estimator - pooled cross sections over time

Consider two time periods, where In time-period 1: a random sample is collected for group 1 (control) and a random sample is collected for group 2 (treated). In time-period 2: a random sample is ...
47 views

### What is the best way to test and validate a multivariate regression using OLS?

I am implementing a multivariate regression from scratch using Ordinary Least Squares to get the weights. I noticed that this method does not have any hyperparameters to tweak, so I am not sure what I ...
192 views

### Expected value of the residuals

How would one prove that the expected value of the residuals from OLS regression is zero? I will make two cases. In the first case I treat $X_i$ as random and in the second case I treat it is non-...
29 views

### ANOVA tables are exactly the same using OLS and Poisson regression?

I have fit two models in R, as follows: m1 = lm(y ~ x * z) m2 = glm(y ~ x * z, family="poisson") Where y is the dependent ...
51 views

### Degrees of freedom correction in estimation of AR(p) process

Assume that I have a process $y_t$ such that $$y_t = c + \phi_1 y_{t-1} + \ldots + \phi_p y_{t-p} + u_t$$ where $u_t$ is i.i.d. white noise such that $E[u_t] = 0, \forall t$ and $E[u_t u_s]$ is equal ...
38 views

### Is fixed effects (within estimator) estimated using OLS?

In a paper I submitted to a journal, I estimated a fixed effects panel model with regional fixed effects. The reviewer of my paper in his comments said my estimation was bad because I "simply used OLS ...
38 views