# Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

1,586 questions
19 views

### Show positive semi-definiteness of co-variance differences

I am having difficulty checking if $$(A'D^{-1}A)(A'D^{-1}BD^{-1}A)^{-1}(A'D^{-1}A) - (A'A)(A'BA)^{-1}(A'A) \succeq 0$$ where $B\succ 0$ is a square co-variance matrix and $D$ is a diagonal matrix ...
38 views

### Is it valid to iterate over every permutation of a regression specification and compute an “average significance?”

I had an idea, and was wondering if it's ever done and, if so, how to do it in an appropriate manner. Let's say I run an OLS model and the results come back significant. There is discussion as to ...
21 views

### Derivation of Formula for linear regression with data points forming a circle

Suppose we have one dimensional data with following conditions.1. The value of this single feature lies in the closed interval [-a,a].2. The associated target value with each data also lies in the ...
11 views

### How to obtain a trend estimate using only ols?

I have data on $y$ and $t$ and I want to estimate the trend in $y$ that is supposedly given by relation $y=a(1-b e^{t/τ})$. I want to use linear regression on some transformed version of the model in ...
300 views

### OLS robust to outliers

I am facing the following problem: I have a training sample and estimate a model on that training sample. My model is simply OLS: $y_t = a + \beta x_t + \varepsilon_t$. The model is estimated on ...
1k views

### Gradient descent doesn't find solution to ordinary least squares on this dataset?

I have been studying linear regression and tried it on below set {(x,y)}, where x specified the area of house in square-feet, and y specified the price in dollars. This is the first example in Andrew ...
14 views

### How would I figure out - TSS of Y, Yi and XiYi from the following information?

I came across an analogous question when revising and have no clue how to approach it. The Given information is ΣXi= 20, ΣYi=40 Σ(Xi-x̅)²= 40, Σ(Xi-x̅)(Yi-nȳ)=20 and n=20. The question requires ...
11k views

### Difference between fixed effects dummies and fixed effects estimator?

I started to read about panel regression models. However, I am a bit confused about the different model specifications in the fixed effects model: Does a fixed effects panel regression always mean ...
16 views

### Cluster-robust standard errors in panel data analysis

In a simple panel data analysis with data on 64 firms over 8 years, I use cluster-robust standard errors (at the firm level) to evaluate significance of coefficients. I observe important differences ...
52 views

### Sequential Least Squares for Tikhonov Regularization [duplicate]

Given a Weighted Linear Least Squares problem where the cost function is given by: $$J = { \left( x - H \Theta \right) }^{T} {C}^{-1} { \left( x - H \Theta \right) }$$ There is a Sequential ...
21 views

### Cannot replicate a 95% CI using: Beta ±1.96*SE [duplicate]

I am having a lot of difficulty figuring out why I cannot calculate a 95% CI by hand with one set of data; but I can using the same process with a different set of data. Drawing on OLS results; I am ...
10 views

### Comparing the output and fit of an OLS and a Tobit model, and comparing Tobit models

I am trying to estimate a quite complicated model (many variables with different structures), with a limited dependent variable, which ranges from 0-100% with about 45% of the sample having an ...
31 views

### Does standard error affected by the coefficient?

I make a comparison on ridge regression and OLS using simulation. As i set my correlation as 0.9, which is high, i expect the standard error of ridge regression to be low. However, it is not. ...
9 views

### Least squares and identificability condition

Let a discrete-time system (which is minimal) with input $u \in \mathbb{R}^m$ and state $x \in \mathbb{R}^n$ be $x_{k+1} = [x^T_k \quad u^T_k]\begin{bmatrix} A^T \\ B^T \end{bmatrix} + v^T_k$ ...
12 views

### upward/downward bias of negative variable

If I have a variable that, considering some omitted factor, should have fallen by a higher amount than when it is not there - would that be a downward bias? I.e. the decrease is not large enough, so ...
27 views

### Degrees of freedom in OLS regresison vs Bootstrap

I understand that in OLS, the degrees of freedom for estimating the variance of the residuals is n-q-1. We loose q+1 degrees because they are "used" to analytically determine the q parameters and 1 ...
11 views

### How to motivate a POLS?

How would you justify the usage of Pooled OLS regression instead of Fixed effects? If I am calculating just correlation between two phenomena, may I get rid of these fixed effects? May I choose to ...
43 views

### Does $\hat{\beta}=(X′X)^{−1}X′ \ Y$ simplify further?

As far as I know $(AB)^{-1}=B^{-1}A^{-1}$ and matrix multiplication is associative so: $$(X′ X)^{−1}X′=(X^{-1})(X')^{-1}(X')=X^{-1}.$$ What am I doing wrong?
32 views

### Obtain global linear regression estimate from subsamples

I want to estimate $\widehat\beta$ in a simple linear regression with scikit. $$y = X \beta + \varepsilon$$ The problem is that the dimension of the complete $X$ is too large to fit into memory. Is ...
3k views

18 views

### Log-normalization of predictors

I have the following dependent and independent variables for my linear regression model. Since they are all in different scales (some of the are % others continuous variables), I was suggested to take ...
19 views

### Does a panel regression model make sense for my data?

This might be a bit of a newbish question, but I recently picked up a forecasting project at my job, and I'm trying to figure out whether it makes sense to run a panel regression like a Fixed Effects ...
26 views

784 views

### Formal errors from non-negative least-squares?

I am computing a standard linear regression subject to a positivity constraint using non-negative least squares (lsqnonneg in Matlab, actually). Is it possible to ...
12 views

### Simulating size distrotion with a VAR(1) model

Firstly, I am not asking for code, I would like the intution of how I would do this. I am testing for size distortion. I have estimated and VAR(1) model and I have the parameters. I want to ...
422 views

### Asymptotic variance of estimated AR(1) coefficient

Consider an $AR(1)$ process \begin{align*} y_{t} + a y_{t-1} &= e_{t} \end{align*} for $t=1,\ldots,N$, where $e_{t}$ is a white Gaussian noise with variance $\sigma^{2}$. How do I express the ...
29 views

### Without testing: when are error terms possibly homoscedastic?

I am facing the following study: In the 1980's, Tennessee conducted an experiment in which kindergarten students were randomly assigned to regular and ...
23 views

### Consistency of an OLS estimator in time series

In the ARMA(2,1) time series $y_t = \beta_0+\beta_1y_{t-1}+\beta_2y_{t-2}+ u_t + \phi_1u_{t-2}$ $u_t$ and $u_{t-2}$ are white noise shocks. The time series are stationary and ergodic. In the ...
534 views

### What is the expectation of the error term conditional on Y?

In an OLS model, we usually assume $(Y_i, X_i)$ are i.i.d., and $U|X$ has mean $0$ and variance $\sigma^2$, and then run a regression of $Y$ on $X$. But when we are running a regression of $X$ on $Y$ ...
22 views

### Residualizing in OLS

Consider a linear regression $$y_i = A_i' \theta + B_i' \psi + \epsilon_i$$ There's a "trick" to find the parameters $\theta$ and $\psi$ in a stepwise fashion. First minimizing squared error wrt....