Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

1,749 questions
Filter by
Sorted by
Tagged with
8 views

What is an example to show Wald's test does not confirm significance at the same level for each variable?

Suppose that the true model is given by Y=0.3+X1+X2+X3. Assume we have 100 training examples where each covariate vector (X1, X2, X3) is randomly drawn from some distribution P and Y is generated ...
7 views

Does endogeneity problem matter when proving the existence of association/causality relationship?

In social science field (particularly Finance and Operations Management), we usually need to prove or disprove hypotheses of the type: X are positively associated with Y. One of the typical method to ...
8 views

What statistical test is most appropriate when my data consist of multiple series, each based on an individual sample?

I'm trying to determine the effect of an interferent $X$ on the measurement of a substance $Y$. Ultimately, I'm looking to predict $Y_{actual}$ within a confidence interval, given $Y_{observed}$ and a ...
14 views

How do the Gauss-Markov Assumptions relate to the OLS Properties?

How do the 5 Gauss-Markov Assumptions below relate to the OLS properties of Biasedness, Consistency and Efficiency if they are violated?
16 views

Can I use subset of a time series if OLS assumptions are failing?

My OLS model build on full data has heteroscedasticity of residuals. Is it allowed to use a subset of the time series to get around this issue? What are the possible implications?
10 views

Calculating a formula for the asymptotic variance of a function of parameters?

Suppose I have a model $y=\beta_1+\beta_2x + u$ with an estimator $\hat{y}=x\hat{\beta}$. Also suppose I want to determine the average of 3 predictions of $\hat{y}$, called $\hat{y^*}$, and then ...
26 views

Perfect multicollinearity with a cubic term in the model?

I'm trying to figure out why adding a cubic term in the model doesn't guarantee a perfect multicollinearity. If $X$ is known, then $X^3$ is known in both magnitude and sign and vice versa. It may not ...
76 views

Question regarding the distribution of OLS estimators

In linear regression: Why does the sampling distribution of OLS estimators depend on the underlying distribution of errors? Thanks! (I am relatively new to statistics in general, so more simply ...
23 views

FGLS vs OLS tradeoff

Hi, Can someone tell me what is the tradeoff between using OLS and FGLS? I know the conditions under which FGLS is more efficient than OLS. But what about robustness? It is usually said that the ...
17 views

Estimating a logistic regression with OLS? [duplicate]

NB: This question is different from this one which assumes that we have computed the LHS of the regression equation with no issue. My question is about how to compute this LHS. Consider a simple ...
23 views

83 views

In an OLS regression, will excluding all data for a non-reference category of a dummy variable impact the other dummy level categories?

Say I have an OLS regression with a dummy variable level A, B, C and D, where A is the reference category. Will the estimated coefficient value and/or statistical significance of B or C change or be ...
23 views

Can I transform a few features to polynomial in multi regression?

Let's say I have 3 features in my ols model as below. model = sm.ols(formula='y ~ a + b + c', data=df) Question 1. If I want to transform the feature 'a' as ...
21 views

AR(1) process can be estimated using linear regression

Can the $AR(1)$ process represented as $$x_t= ax_{t-1}+\epsilon_t$$ be estimated by regressing $x_t$ on its lagged value $x_{t-1}$.
24 views

What is the process of obtaining Var(βhat) in simple linear regression?

I have just started statistics and we have used the estimation strategy OLS to obtain the parameter estimate of the independent variable for a simple linear regression model. As I understand it, my ...
37 views

Neural Net Regression SSE Loss

Notation $y_i$ is observation $i$ of some response variable $Y$. $\hat{y}_i$ is the value of $y_i$ predicted by the regression. $\bar{y}$ is the average of all observations of the response variable....
42 views

Unbiasedness of OLS estimator

Consider a very simple linear regression model with following assumptions: 1) No assumptions on how $x_i$ is generated (assume random design, not necessarily IID); 2) $\mathbb{E}[\epsilon_i|x_i]=0$, ...
51 views

Should the existence of a closed-form solution inform the choice of robust regression method?

Suppose one has a linear least squares problem of the form \begin{align} \xi^* = \textrm{arg min} \ \sum_{i = 0}^n \left \lvert \ {\bf v}^T({\bf x}_i) \ \xi - c({\bf x}_i) \ \right \rvert^{\ 2} \in ...
44 views

Relationship between SSE with GLS and OLS

I have been trying to derive if there is any relationship between the sum of squared residuals (SSE) from a model estimated with GLS, and the same model estimated with OLS. Professor Chung-Ming Kuan, ...
20 views

Unbiased estimate of variance for 1d linear regression

I'm trying to show for the simple 1-D linear regression case that the estimate $\frac{1}{n-2}\sum_{i=1}^n \hat{\epsilon_i}^2$ is an unbiased estimate of $\sigma^2$, which is defined as the variance of ...
30 views

LASSO Regression with noise

I know LASSO regression is useful to exclude redundant features, so can it be useful when you have noisy data? I explain better with this example: Suppose I generated a data set using an equation (e....
31 views

13 views

What is the difference in meaning between the Pearson Coefficient and the error from a least squares regression line?

From what I understand, the Pearson Coefficient determines if there is a linear relationship between some data. However, doesn't the error from a least squares regression line also do basically the ...
25 views

OLS reinterpretation (?)

I have read on a book written by a professor something similar to the following and want to check this statement on the forum, since it is the first time I have heard it. I have read that the OLS ...
45 views

Standard error of the intercept in Frisch-Waugh theorem (de-meaned regression)

I am applying in Frisch-Waugh Theorem to partial out a set of fixed effects D and get the OLS estimates and standard errors of the remaining regressors ...