# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

1,586 questions
37 views

### Kernel and regularization parameter of James–Stein estimator

Consider a FIR model of the form $y= Ug_0+e$ with $e$ white noise with variance $\sigma^2$. We assume that we have collected N input-output measurements $y$ and $U$. The James–Stein estimator is ...
47 views

32 views

### Heteroscedasticity and weighted least square estimator

"In presence of heteroscedasticity, OLS estimators are unbiased but inefficient" Showing the unbiased part is relatively easy. Some authors have explained the inefficiency with the help of new ...
16 views

44 views

### How to solve a well fitted model - Model Misspecification

I am currently writing a paper, analysing the impact of goldprice movements on the capital structure of gold mining firms. My basic model is a simple OLS model with (y=leverage and x=ln(goldprice)). ...
18 views

61 views

### Least Squares with respect to a Matrix

There is a nice geometric interpretation of ordinary least squares where $\beta \in\mathbb{R}^p$ is to be learned/ estimated from data given by $y \in\mathbb{R}^n$ and $x \in\mathbb{R}^{n\times p}$ ...
20 views

### Standard errors of OLS estimate if regressor is a stochast?

Assume the model classical linear regression model (with for simplicity only one regressor) $$y=X\beta +u,$$ with $u$, $X$ independent, and $\operatorname{Var}(u|X)=\sigma^2I_n$. Assume for ...
136 views

### Adding a Constant to Every Column of X (OLS)

In OLS, if I have design matrix X (an NxK matrix of full column rank) and I add a constant, such as 2, to every entry of X, how does that change my estimators? Let's denote $\tilde{X} = X + 2$. I ...
240 views

50 views

### Why is $\mathbb{E}\left[\frac{1}{n-2}\sum\limits_{i=1}^n e_i^2\right] = \sigma^2$?

The context that we are presented with a linear model $$Y_i = \beta_0 + \beta_1X_i + \epsilon_i,$$ where $\epsilon_i \texttt{~} \mathcal{N}(0,\sigma^2)$. We obtain predictions for $X_i$ by plugging ...
32 views

### Intepretting Linear Mixed Effects Model Output

I'm trying to reconcile the output of this MixedLM Model Output with my knowledge of OLS model outputs, e.g., MixedLM has Z-Stats vs. T-Stats for OLS. Model code for reference, but this is not my ...
66 views

### Obtaining the possible least squares solutions when $X^TX$ is not invertible

If $X^TX$ is not invertible, what is the set of solutions for the least squares estimator $\hat{\beta_1}$ in the below? $Y_i = \beta_0 +\beta_1(x_i-\bar{x}) +\epsilon_i$ I got as far as writing out ...
60 views

### Cointegration regression for stationary series

I have seven variables that are stationary at first difference. I am trying to figure out how variable X influences variable Y, conditional on Z, W, A, B, and C. Which would be more appropriate to ...
32 views

### Interpretation of Coefficients in Semilogarithmic Interaction Regression Model

The linear model for consideration is a binary interaction multiple regression model using OLS. The dependent variable is household net worth transformed with the inverse hyperbolic sine ...
19 views

### Questions about Endogeneity and IVs

I am doing the returns of education on earnings. I have education (number of years of edu) as my endogenous variable and I am using Father Education as an instrument. I am using SAS and I ran a 2sls. ...
8 views

### How to Observe OLS Effects Across Time Spans?

Forgive me if this should be in the web chat, I don't have the reputation yet. I have a data set that spans across roughly 10 years: ...
109 views

### Linear Regression: Why variance of β is high when $X^TX$ is singular

I know how to derive β when $X^TX$ has an inverse and on that condition, β is an unbiased estimate of β* with mean 0 and β* and variance $σ^2(X^TX)^{-1}$. But why the variance will become very high ...
242 views

### What is the general guideline for dropping dummy variables in a regression model?

For an ordinary least square (OLS) regression problem of the form $y = \beta + w_1 x_1 + \ldots + w_n x_n$, let's say I have 3 categorical variables. gender: male, female type: office, field, manager ...
29 views

### Show $T_0^2$ (t-statistic) is F-statistic

I am trying to show that $T_0^2 = F$ (but I just can't get it), where T_0^2 = \left(\frac{\hat{\beta}}{\sqrt{\frac{\hat{\sigma}^2}{\sum_{t=1}^n (x_t-\bar{x})^2}}}\right)^2 = \frac{\...
24 views

### OLS difference between individual level and aggregate level estimates

I am new here, so please accept my apologies if this does not belong here. My question essentially concerns what appears to me a special case of the ecological fallacy. Consider a simple model that ...
63 views

### Least Squares fit of model - R

The data file (X in code thread below) contains the record of monthly data X[t] over a twenty year period. The data can be modelled by X[12j+i] = Mu + s[i] + Y[12j+i] where (i=1,...,12; j=1,...,k) ...
35 views

### Direction of Bias in OLS with systematic measurement error

I am looking at a study that wants to measure the effect of $x_t$ on $Y_t$, but the true values of the $x_t$ are not observed. Instead, what is observed is a minimum value for $x_t$. In effect, what ...
96 views

### Expression for $\hat{\beta}$ in simple linear regression

For simple linear regression, I have in my notes that $\hat{\beta} = \frac{\sum(x_i-\bar{x})(y_i-\bar{y})}{\sum(x_i-\bar{x})^2}=\frac{\sum(x_i-\bar{x})y_i}{\sum(x_i-\bar{x})^2}$ (was intended as a ...