Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

0
votes
0answers
17 views

Non-normal regression errors: consequences and solution

Doing an empirical econometrics project, I have managed to confuse myself on one of the basics and wonder if you could lead me on the right track. What I wonder about is the normality assumption of ...
0
votes
0answers
6 views

Regress in two variables against basis functions when one of them might be “sparse”

I'm tasked with the problem of fitting a series of functional forms $\{f_t(S,K)\}$ along the time axis. At each time $t=0,1,\cdots,T$, there are $N_t$ samples $$(F_{t,i}, S_{t,i}, K_{t,i}),\quad i=1,2,...
1
vote
1answer
20 views

Updating regression solutions for removing a regressor without the original dependent variable

Note: This question is analagous to the question I asked here except instead of adding a column, I am removing it. I am interested in a linear regression on the model; $Y= X\beta + \epsilon$ And I ...
2
votes
1answer
63 views

Updating regression solutions for a new regressor without the original dependent variable

Note: This question is analagous to the question I asked here except instead of a removing column, I am adding it. I am interested in a linear regression on the model; $Y= X\beta + \epsilon$ And I ...
0
votes
0answers
15 views

Statistical significance of co-efficient derived through several regressions for panel data

For the panel data that I have, I'm trying to run multiple one-variable regressions (y=ax+e) across time periods - i.e. one for each time period. Naturally, the a's that I find from each regression ...
0
votes
0answers
30 views

Testing interaction terms individually and simultaneously

I am currently testing two interaction terms individually in OLS regressions. Both interaction terms are significant (p < .5) when tested individually (i.e., two regression equations, one for each ...
0
votes
0answers
24 views

What is a computationally tractable way to infer statistical power in the context of OLS with robust clustered standard errors?

I have clustered data and am trying to scale up OLS with robust clustered standard errors (implemented in Python Statsmodel) as the default analytical framework going forward. It is important that I ...
0
votes
0answers
22 views

Finding inverse of $X'X$ in the case of two regressors [duplicate]

Variance of OLS etimator in matrix form look like this: $Var(\hat{\beta_j})=\sigma^2(X'X^{-1})$ I'm struggling to derive inverse matrix for the case with two independent variables. $X'X$ $=$ $\...
0
votes
0answers
24 views

MLE estimate for least squares if features have Gaussian noise

We have come across the problem of MLE estimate for least squares if errors are normally distributed, eg, $Y_i=\beta x_i+\epsilon$, where $\epsilon$ ~ $N(0,\sigma^2)$. The estimate for the above case ...
1
vote
0answers
9 views

Interaction in logit

I am estimating a model where interaction terms play a role, and I am wondering which specification I should use, and how to interpret the results. More specifically, I regress a binary variable, say ...
1
vote
1answer
33 views

OLS loss function 3-d surface plot

I was trying to plot the OLS loss function as a function of coefficients $\beta_0$, $\beta_1$. As far as I know it should be a convex function with one local minimum which is also a global minimum. I'...
0
votes
0answers
13 views

Direction of bias when changing from OLS to IV

If when using an instrumental variable it increases the size of the coefficient and changes the direction of the relationship compared to OLS --> what direction of bias does it suggest in the OLS ...
0
votes
0answers
18 views

Heteroskedasticity in linear probability models

I have a class on linear probability models. We want to estimate a model $y=\beta x$ where both $y$ and $x$ can be either $0$ or $1$, so that the conditional expectation function can be expressed in ...
1
vote
0answers
22 views

Fixed Effects model

I am trying to understand the fixed effects model. $ Y = D\alpha + X\beta + \epsilon$ Where $\alpha$ is the fixed time invariant effect. If we model this using Frisch Waugh lovell partitioning ...
1
vote
0answers
22 views

hypothesis testing doubt

I have the OLS regression model: $Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_3 + \beta_4X_4 + \epsilon$ I want to check the hypothesis: Ho : $\beta_2*\beta_3$ = 1 Will I use the Delta ...
0
votes
0answers
23 views

OLS Heteroskedasticity correction

I have a data and was trying to correct for heteroskedasticity (which is significant as per Breush Pagan test). However, after using the robust command in stata, my standard errors of almost all the ...
1
vote
1answer
31 views

Validity of the linearity assumption

I have to research unaided recall of commercials given a set of variables. So, I formulated the following model: $unaided = \beta_0 + \beta_1duration + \beta_2blocksize + \beta_3position + \beta_4 ...
0
votes
0answers
9 views

F test, Model selection test

How to compare which of the following two models is a better fit: M1 $y$ = $\alpha$$X$ + $\epsilon$ M2 $ln(y)$ = $\alpha$$X$ + $\epsilon$ Can we run a test statistic to compare the two models? ...
0
votes
0answers
13 views

How to test for sample selection bias in heckit estimation?

I'm doing a heckit two step estimation by hand where i first estimate the labor force participation, compute the inverse mills ratio by using the linear predictors of the model and add it as an ...
0
votes
1answer
17 views

Prove that the direction in Least Angle Regression makes equal angle with all predictors [closed]

Least Angle Regression iteralively adds predictors according to the procedure described here : Writing by hand first steps in Least Angle Regression (LARS) We note $A_{k}$ the active set of variables ...
0
votes
0answers
21 views

Singular values for a latent-factor model

Suppose we build a latent-factor model using alternating least squares (ALS) or stochastic gradient descent (SGD). Can we calculate weights for each latent factor, in a similar way to how the singular-...
2
votes
1answer
155 views

In the presence of heteroskedasticity, is quantile regression more appropiate than OLS?

..for understanding the relationship between a dependent and independent variables, given that quantile regression makes no assumptions about the distribution of the residual.
0
votes
0answers
26 views

How to interpret plot residuals vs fitted values?

I run a ols regression and want now check the linearity assumption. I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
0
votes
0answers
13 views

Proof for multicollinearity consequence

I came across this statment "Even extreme multicollinearity (so long as it is not perfect) does not violate OLS assumptions. OLS estimates are still unbiased and BLUE (Best Linear Unbiased Estimators)"...
0
votes
0answers
7 views

Why different seeds produce different mse values for regression tree and ols?

I compare regression tree and ols in terms of out of sample prediction. I realized that the mse values changed when i change the seeds before getting train and test set. Sometimes ols is better ...
0
votes
0answers
27 views

OLS Explained Variance

I'm using simple OLS and I want to get how much each independent variable explains variance of dependent variable. Is it possible in context of OLS or I need to use other methods?
0
votes
0answers
16 views

OLS estimator - caculating mean and variance [duplicate]

Yi = a + Xi + ui (i=1,2,...,n) In this regression model(there is no b(Coefficient of Xi)), I calculated the estimator of a. a hat = ΣYi/ΣXi And then I tried to calculate E(a hat) and Var(a hat) ...
0
votes
1answer
37 views

When does OLS Regression outperform regression tree in term of out of sample prediction?

In my Master thesis i compare ols regression to regression tree to predict wages. I thought that i will get better prediction with the regression tree because it cathes more interactions. But now i ...
0
votes
0answers
26 views

Is there any benefit of using GLS when the regressors are identical

I am reading Greene, Econometric Analysis, 7th Addition, I am seeking a point of clarrification. "The case of identical regressors is quite common [think a VAR mode].... In this special case, ...
0
votes
0answers
4 views

interaction term between continuous moderator and independent dummy variable (in OLS)

i would like to check if there is an interaction between the independent contiunous variable leverage and the choice of auditor - dummy variable (audited by big4: 1). The dependent variable is ...
1
vote
1answer
51 views

Estimating a VAR using OLS vs GLS

I have read in several places that I can estimate a VAR model equation by equation using OLS instead of using GLS, if I have the same explanatory variables. Do I need to make any assumptions about ...
1
vote
0answers
38 views

Estimating a VAR model via OLS

I am looking at Vilasuso (2001), who says that when using least-squares to estimate causality in mean, there is significant size distortion if the conditional variances are correlated. My question ...
0
votes
0answers
10 views

Heteroscedasticity in VAR Residuals and Least Squares

If I have a VAR model, think of the simple case with two variables $y_1$ and $y_2$, Vilasuso (2001): says that if the conditional variances of $y_1$ and $y_2$ are correlated, significant size ...
0
votes
1answer
19 views

How does correlated relationship implies non zero conditional mean?

i.e. if $cov(X,Y)\neq0$, how do we derive $E(X|Y)\neq0$ it seems that it is quite obvious and usually be taken into granted, but I just couldn't figure it out in a short time, can somebody give me a ...
2
votes
2answers
86 views

Predict a vector of values with constraints? [duplicate]

I am aware of a variety of methods for simultaneously predicting multiple outcomes known sometimes as multivariate regression/analysis. However, my situation is a little more special. I am trying to ...
0
votes
0answers
10 views

How to determine parameter (co)variance of multiple output regression

I am trying to identify parameters of a model. Pratical application: in a robot, to estimate physical parameters (length, mass, inertia, etc.) of a multi-degree of freedom (DOF) non-linear dynamic ...
1
vote
0answers
11 views

Line of Best Fit

If we have a dataset with two variables, X & Y, we can find the line of best fit using the empirical data (and whatever method suits you best). However, what if know the true joint distribution ...
1
vote
0answers
16 views

Different variables for different subgroups of linear model

Let's say I have a multiple linear model for a population of the form $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + e$ There is a third variable I want to introduce that I believe significantly explains ...
0
votes
0answers
15 views

IID random sample assumption in classical OLS regression

In classical OLS regression, I get that the assumptions: Strictly exogenous errors $\implies$ unbiased OLS estimator Errors are normally distributed $\implies$ allow to make tests (t-tests, etc.) ...
0
votes
0answers
7 views

Construct least-squares ultrametric (hierarchical clustering) fits to doubly-standardized “flow” tables and compare to single-linkage-type fits

Figure 1 of the paper, "Hierarchical Migration Regions of France" (IEEE Transactions on Systems, Man and Cybernetics, 4 (1976) 321-324) (https://www.researchgate.net/publication/...
1
vote
0answers
27 views

Does it make sense to use bagging with least square regression? if so, why?

We know that bagging can reduce the variance of the estimator, while keep the bias around the same. If we use bagging to ensemble multiple least square regressors, then are we going to reduce the ...
1
vote
1answer
37 views

What is the meaning of “unexplained” in unexplained variance or residual sum of squares?

I understand the formula of RSS and RSE but it confuse me every time I read unexplained variance. I don't understand why the term unexplained is use.
2
votes
1answer
26 views

Linear Regression in stats model OLS

After removing the insignificant variables(p-value >.05), I fitted the OLS model again. I found there are still many variables which had p-value < .05 earlier have p-value > .05 now. Do I need to ...
1
vote
1answer
23 views

Split by gender, or pool them into the same model?

Could someone please explain to me in layman's terms the difference between the following 3 scenarios in OLS multiple regression? Let's assume my DV is income, and my IVs are White race, Asian race, ...
0
votes
1answer
42 views

Is the optimal lag length for the Hansen and Hodrick and Newey West robust standard errors the same?

Is the optimal lag length for the Hansen and Hodrick and Newey West robust standard errors the same? I have read in Greene that the optimal is $T^{1/4}$ for Newey-West, is this the same for Hansen ...
0
votes
0answers
74 views

Given $Y_{i} = \beta_{0} + \beta_{1}x_{i} + \epsilon_{i}$, prove $\beta_{0}$ and $\beta_{1}$ are uncorrelated iff $\overline{x} = 0$

Let $Y_{i} = \beta_{0} + \beta_{1}x_{i} + \epsilon_{i}$ $(i = 1,2,\ldots,n)$, where $\textbf{E}[\epsilon] = 0$ and $\textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n}$. Find the least square estimates ...
0
votes
1answer
29 views

least square with equality constraint and singular matrix

I have the following problem to solve: Ax = b where A is singular. To resolve this I introduce a condition Cx = d Even after this, I am not able to solve it using scipy.optimize.minimize as it ...
0
votes
1answer
55 views

Get lower bounds and upper bounds of a curve

I have fit the wave data as below using Least Square optimization and used Root Mean square to calculate upper and lower bounds. I wanted to know is there any other alternative exists which gives a ...
0
votes
1answer
35 views

Changing units in linear regression

A linear model: $y_{i}=\alpha +\beta x_{i}+u_{i}$ was estimated using least squares and the following estimate was calculated: $\widehat{\beta }=\frac{\sum (x_{i}-\overline{x})y_{i}}{\sum (x_{i}-\...
0
votes
1answer
45 views

Should I be using pooled OLS or Fixed effect regression for my model here and why?

I have run two different regression models, one fixed effect and one pooled OLS on my data. my data looks at the number of visits to hospital regressed over age, marriage, income, insurance etc. I ...