Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

0
votes
0answers
23 views

Random effects model vs pooled OLS

Can someone please explain why bother using random effects if the unobserved constant effects are assumed to not be correlated with the explanatory variable? Why not just using a pooled OLS?
0
votes
1answer
41 views

How to check for independence of errors?

How to check for independence of errors in OLS regression? Let's say I have 10 observations for each hour. If I plot residuals ordered by time, I have the problem that adjacent residuals refer to the ...
1
vote
0answers
10 views

Autocorrelation with Replicates

How to perform autocorrelation with replicates? For each day I have many observations and I want to check wether these observations are correleated with the next day and the day after this day. If if ...
2
votes
2answers
284 views

Why use OLS when it is assumed there is heteroscedasticity?

So I'm slowly going through the Stock and Watson book and I'm a bit confused on how to deal with the issue of homoscedacity/heteroscedacity. Specifically, it is mentioned that economic theory tells ...
-1
votes
0answers
12 views

Independence of Residuals: Multiple Measures for each Point in Time

One assumption of OLS regression is independence of residuals. I'm not sure how this assumption can be checked for the following study design. Every day 5 measurements were carried out, for which I ...
0
votes
0answers
32 views

VAR/VEC model selection

I want to model a relationship between a good's price and a few variables using time-series data. I run VEC/VAR models and get a series of equations. My question is how to use these results (using ...
0
votes
0answers
16 views

Estimating Fourier parameters using least squar emethod

I'm given that $$ \sum_{t=1}^N \epsilon^2(t) = \sum_{t=1}^N\left[y(t) - \sum_{n=0}^{N/2}\left\{a_n\cos\left(\frac{2\pi nt}{N}\right) + b_n\sin\left(\frac{2\pi nt}{N}\right)\right\}\right]^2 $$ and ...
1
vote
1answer
30 views

Bias of omitting squares and interactions

With $x_1\sim N(\mu,\sigma^2)$ and a population model... $Y=\alpha_0+\alpha_1X_1+\alpha_2X_1^2+\epsilon$ ...if I run OLS omitting the square term... $y_i=\beta_0+\beta_1x_{1,i}+u_i$ ...the $x_1$ ...
0
votes
0answers
19 views

Calculate 95% confidence interval for profile likelihood

I'm not at all a statistician, so please bear with me. I have a mathematical system $x' = f(x,P)$, where $P$ is the set of parameters that I try to estimate and $x = (x_1,x_2,x_3,x_4,x_5) \in \mathbf{...
0
votes
1answer
33 views

Minimize Logged Sum of Squares?

When numerically maximizing the likelihood function it is standard practice to do this indirectly by minimizing the negative log-likelihood. When numerically minimizing the residual sum of squares (...
0
votes
0answers
33 views

Connections between pseudoinverse, linear regression, BLUE, ordinary least square

They all take similar forms, why is that, what are the connections here? pseudoinverse: $Ax=b, x=(A^TA)^{-1}A^Tb$ linear regression: $ \hat{y}=x^T(X^TV^{-1}X)^{-1}X^TV^{-1}y$, where X is the data, y ...
0
votes
0answers
29 views

Before-During-After Regression Analysis

I am currently trying to study the effect of the prolonged absence of the MVP in a basketball team on the performance of the team as a whole. Specifically, the idea was to create a regression model ...
1
vote
1answer
39 views

Endogeneity - Omitted variable bias in OLS

If a have a true model $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon$ but $x_3$ is unobservable. What are the consequences of having a unobservable variable which correlates ...
0
votes
0answers
25 views

Performing a cross-sectional regression

This might seem a very fundamental question, but I am having issues grasping how a cross-sectional regression is performed? I have approx. 40 million data points over a period of 21 years, wherein I'd ...
1
vote
0answers
36 views

Linear least squares algorithms

I have stumbled across these two questions and accepted answers: (1) Do we need gradient descent to find the coefficients of a linear regression model? (2) Why use gradient descent for linear ...
1
vote
1answer
43 views

Diff-in-Diff special type of OLS?

Is difference-in-differences just a special type of OLS? Can I add fixed effects in my diff-in-diff model?
1
vote
0answers
35 views

Lagged Dependent variable in OLS

I have a question about one of my models. I am sorry if I am using Terms wrongly, as I am part of the management research field and this quite often leads to different terminologies. I try to model ...
3
votes
1answer
87 views

Errors and residuals in linear regression

I think in common literature about statstics the authors are often very imprecise when it comes to residuals and errors. So far, I could not work that difference out completely and therefore have ...
1
vote
0answers
19 views

How to compute the “hat-matrix” of constrained least squares

I'm attempting to calculate the studentized residuals on a (equality) constrained least-squares regression for outlier detection. However, i'm a little uncertain on how to calculate the leverages, $...
1
vote
0answers
15 views

Partially-Linear Least Squares

Model I'm working with univariate time series data $(y_1, \dots, y_n)$ where time $t \in [1, n]$. Suppose the mean function has a known form, in my case $$\mathrm{E}(Y_t) = 1 - \alpha e^{-t/\tau_1} - ...
2
votes
2answers
73 views

When do you use logistic regression vs. when you do use OLS?

If you are creating a regression model where the response variable is a numerical value, but one of the variables is a dummy (binary), can you use OLS-method? Do you only use logistic regression if ...
2
votes
3answers
224 views

OLS effect of X squared ond dependent variable

I have my OLS regression: $y = \beta_1 + \beta_2 X_2 + \beta_3 X_3 +\beta_4 (X_3)^2$ Could anybody please explain to me the effect of a change in $X_3$ on the dependent variable?(Is the effect ...
0
votes
0answers
38 views

Can we unify generalized linear models and ordinary least squares by switching between two metric spaces

Lots of smart people out there. Maybe someone has seen this concept. In linear regression using ordinary least squares (OLS) we simply project the response Y onto the range of the design matrix X. ...
0
votes
0answers
7 views

how to systematically choose the thresholding in least square solver, for solving ill-posed least square problem?

I have a system $Ax =b$, where $A\in\mathbb{R}^{300\times 200}$, but $rank(A)=70$. And I know the true solution. I tried standard least square solver such as ...
2
votes
1answer
32 views

Is constrained (nonnegative) least squares a form a regularisation?

Is nonnegative least squares already a form of regularisation? By adding a constraint that $\beta \geq 0$ (the coefficients), does it make sense to add another regularisation term as in LASSO or ridge ...
6
votes
1answer
68 views

What is the difference between $E[\varepsilon\mid X]=0$ and $E[\varepsilon X]=0$ in OLS regression?

Why is the assumption $E[\varepsilon X]=0$ weaker than $E[\varepsilon\mid X]=0$?
0
votes
1answer
56 views

Showing that the minimum-variance estimator is the OLS estimator

Recap of required theory Consider the following regression: $$y_i = \alpha + \beta x_i + u_i \tag{1}$$ where $y_i$ are iid and $x_i$ are deterministic (i.e. fixed). We know that the OLS estimator $...
0
votes
1answer
40 views

How to use White-test to check if the heteroscedasticity has been effectively dealt with by a WLS?

Consider an OLS model with $n$ observations and $p$ explanatory variables (including an intercept term) $$y=X\beta + \epsilon$$ We may use a White test to (approximately) check for the presence of ...
2
votes
1answer
63 views

Transform data while keeping the mean constant

I am using an OLS regression to fit a model to some data. The estimated response is given by the usual $$\mathbf{\hat{y}} = \mathbf{Py} = \mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}...
4
votes
1answer
188 views

Sufficient Statistic for $\beta$ in OLS

I have the classical regression model $$y = \beta X + \epsilon$$ $$\epsilon \sim N(0, \sigma^2)$$ where $X$ is taken to be fixed (not random), and $\hat\beta$ is the OLS estimate for $\beta$. It is ...
0
votes
0answers
49 views

Including both state, time and state-time fixed effects in a regression

My question relates to the use of interacted fixed effects (say suburb-time) together with non-interacted fixed effects (suburb, and then time) in the same regression. Specifically, imagine we had a ...
2
votes
0answers
78 views

Leave-one-out cross validation for a linear model through the origin

An Introduction to Statistical Learning by James et al. defines the leave-one-out cross-validation estimate for least-squares as: $$CV_{(n)}=\frac{1}{n}\sum_{i=1}^n \biggr(\frac{y_i-\hat y_i}{1-h_i}\...
2
votes
0answers
40 views

Linear Prediction and Linearity of CEF

I am revisiting the basic notions of linear regression and stumbled upon the following idea in Cameron and Trivedi's Microeconometrics book: However, for the conditional mean to be linear in x, so ...
0
votes
1answer
26 views

Is there a systematic reason why a model trained on a subset of data does better out-of-sample than the same model trained on the full dataset?

I trained a linear regression model using 3000 data points. (OLS regression, no regularization.) Then I trained another model with the same predictors (about 25), but with a subset ($n=700$) of the ...
3
votes
1answer
58 views

Why is individual R-squared higher than overall R-squared?

I ran a ridge regression model on a set of data across 6 groups. As you can see, the overall R-squared is low. Because groups A and B make up the most of the data, I would think the overall R-...
1
vote
0answers
46 views

What statistics can I use to compare OLS to an ordered probit

I am trying to justify to the use of an ordered probit, my dependent variable is a survey response on a likert scale so is likely ordinal, but I wanted to provide a goodness of fit stat to back up my ...
0
votes
0answers
17 views

Interpretation of Economic Significance of Coefficients in Linear Probability Model and Reversing the Dependent Variable

I have the following regression: open_regime = B1 GovernmentSupport + B2 ln(R&D Budget) + other_controls + FirmFE + TimeFE + ProjectCategoryFE where ...
5
votes
2answers
126 views

calculating the p-values in a constrained least squares

I have been using Matlab to perform unconstrained least squares (ordinary least squares) and it automatically outputs the coefficients, test statistic and the p-values. My question is, upon ...
0
votes
2answers
54 views

ANOVA multicollinearity adjustment

I am using the statsmodel.ols module to compute an omnibus (ANOVA) F-test for three within-subjects factors; 2*3*2 levels design. The Cond. No. of the omnibus test (...
1
vote
1answer
24 views

Regression doubt on ols estimator

I understand that beta-1 and beta-2 are parameters/OLS Estimators. But why is the variance of the error term a parameter?
1
vote
0answers
66 views

Fast way to obtain SSR (Sum of Squares residuals) from QR in least square model?

I am using a linear regression, yet the only output I need is the Sum of Squared Residuals (SSR), I don't care about the coefficients. (Context is a non-linear LS, which is linear given an extra ...
1
vote
2answers
66 views

Linear model with two columns where one column is a linear transformation of another

If we have a linear model $y_t = β_1x_{1t} + β_2x_{2t} + e_t$ where the errors $e_t$ are i.i.d normally distributed: $e_t$ ~ $IIN(0, σ^2)$, t = 1,...,T. The vectors of random regressors $X_1$ and $...
2
votes
1answer
41 views

In OLS, while using log-log and linear-log transforamtions, is valid to transform some regressors only?

In OLS I was wondering if it is valid to log-transform some regressors only. Specifically, continuous regressors, because it is advised not to transform binary or categorical variables. For instance, ...
4
votes
1answer
69 views

VIF Drops Significantly When I Delete Some Dummy Variables

Is my model valid even with the high VIF? Does it matter which dummy variable I drop as the reference point? I have a a category variable (Fruit) that I converted ...
1
vote
2answers
50 views

Variance of $\hat{\beta}$ when assuming heteroskedastic error terms

I am wondering why, when we are assuming heteroskedastic error terms: $Var(\hat{\beta}|X) = (X'X)^{-1}X'E[\varepsilon \varepsilon'|X] X(X'X)^{-1}$ simplifies to $Var(\hat{\beta}|X) = (\sum{x_ix_i'...
2
votes
0answers
50 views

Robust Regression in MATLAB's robustfit: what is the optimal weight function to tackle heteroskedasticity?

I'm currently performing a linear regression analysis and encountered a fair amount of heteroskedasticity. Increases in predicted values go along with decreases in residual variance. Otherwise, the ...
0
votes
0answers
56 views

Stuck on a term in $\operatorname{Var}\left[ \widehat{\beta}_0 \right]$ proof

So I was trying to prove that $\operatorname{Var}[\hat{\beta}_0] = \dfrac{\sigma^2n^{-1} \sum{(x_i)^2}}{\sum{(x_i-\bar{x}})^2}$ And I got stuck with the part $\dfrac{-2\bar{x}}{\sum{(x_i-\bar{x})^2}} ...
0
votes
0answers
10 views

Determining the Relationship Between Monte Carlo Breaks and Model Volatility

I'm looking for a statistical test to understand the relationship (if any) between the model volatilities of a stochastic process, and the occurrence of a'break', defined as an instance when an ...
0
votes
1answer
64 views

Expected value of squared least squares estimator

I am trying to prove $E(\hat{\beta} '\hat{\beta}) = \beta'\beta+\sigma^2 *\sum_{k=1}^K\lambda_k^{-1}$ where $\lambda_k$ denotes the eigenvalues of the matrix $(X'X)$ with dimensions $K\times K$. $\...
0
votes
0answers
26 views

Regression equation passing through the origin

I'm going through Multiple Choice Questions of Basic Econometrics by Gujarati. There is a question which states that: It is a simple two-variable regression: Any regression equation written in its ...