Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
1answer
43 views

About stepwise regression and correlation

I am trying to fit a model to some observed data with the least squares method. Now, I am at the stage where I have run a stepwise regression (traditional), with Entry level $=0.025$ and Stay level ...
8
votes
2answers
136 views

How do residuals relate to the underlying disturbances?

In the least squares method we want to estimate the unknown parameters in the model: $$Y_j = \alpha + \beta x_j + \varepsilon_j \enspace (j=1...n)$$ Once we have done that (for some observed ...
0
votes
0answers
6 views

Trouble with horner function in MATLAB [closed]

I have the following homework question: Apply linear least squares with the two models S1(A, B, C) = Ax^2 + Bx + C and S2(A, B, C, D) = Ax^3 + Bx^2 + Cx + D to the data set (0, 4), (1, −1), (2, ...
0
votes
0answers
29 views

OLS derivation question [duplicate]

How come I always see the derivation of $\hat{\beta}$ in OLS using matrix differentiation and solving for when the derivative is $0$. Couldn't one just derive it also by noting that in $Y = X\beta + ...
0
votes
0answers
9 views

Fitting strategy for nonlinear least squares with penalty

I have a model I'd like to fit which has the form $Y = \alpha + \beta_1 * f(x_1; \theta_1) + \beta_2 * f(x_2; \theta_2)$ The $f$'s are complicated, nonlinear, and non convex transformations applied ...
1
vote
1answer
35 views

Why does this have zero sample correlation?

http://web.stanford.edu/~mrosenfe/soc_meth_proj3/matrix_OLS_NYU_notes.pdf On page 4, it says that $x_k'e = 0$ implies that each regressor has zero sample correlation with the residuals. I don't see ...
0
votes
0answers
16 views

creating an indexed dummy variable as a predictor in OLS

I am performing on OLS with two predictors and a response variable. The data is a time series of 450 days approximately. There is an irregular pattern in my response variable - it sometimes ...
2
votes
0answers
14 views

How is the use of OLS in estimating ARCH(q) models justified?

So when estimating ARCH(q) model, some procedures go like estimating first AR(q) model using OLS and then using OLS again against error terms of AR(q). But ARCH suggests heteroskedasticity, and OLS ...
1
vote
1answer
31 views

Hypothesis test for the response variable in a least squares regression model

I have an equation where time it takes to get to work is based on time it takes to depart, number of red lights hit, and number of trains you encounter. The model is shown below: ...
0
votes
1answer
13 views

Comparing how good models with binary dependent variable are

So I'm not very well versed when it comes to statistics, so this is a bit over my head. I can't go into too much detail (NDA, etc etc), but I have four different datasets where in every dataset there ...
0
votes
1answer
61 views

Weighted least-squares negative fitted values

I am running a weighted least-squares regression (where all weights are strictly positive), where my dependent variable is a cross-section of variance values. Since variance is always positive (>=0), ...
4
votes
2answers
122 views

How to know if “best fit line” really represents known set of data?

I have a known set of data. I have created a "linear best fit line" for that set of data. Is there a way to determine how well my set of data fit that best fit line (some sort of score)? I'm very ...
0
votes
0answers
7 views

Performing OLS with gamma transformation

In some specific areas it is common to perform OLS regresion with beta distribution transformation. The α and b parameters are calculated by the sample's μ and σ^2. While the transformed dependent ...
1
vote
1answer
37 views

Weighted Least Squares Through The Origin

For a regression model through the origin, with Var($e_i|x_i) = x_i^2σ^2$ . The corresponding regression model is $Y_i$ = $\beta$$X_i$ +$e_i$. How do I create a least squares model? I know I need to ...
0
votes
0answers
17 views

OLS and multiple hypothesis testing

I have a multivariate regression with more than two independent variables. However, I am only interested in two particular variables. So for simplicity, I write this model in terms of the two ...
3
votes
1answer
25 views

Expected value of least squares estimator $\hat{\beta}$

Given $\hat{\beta} = (X^{T}X)^{-1}(X^{T}Y)$, how do you derive the expected value? I found answers for finding the variance matrix but not the expected value. Thank you kindly.
2
votes
0answers
27 views

Aggregate variance for ordinary least squares?

I am writing some MapReduce code to calculate ordinary least squares from a sample of data. I'd like to include standard error, but I am running into a problem in calculating the variance of the ...
1
vote
1answer
29 views

Consistency of heteroskedasticity-robust standard errors

Is the following statement true or not? When applying OLS with model $y=a+bx+u$, the heteroskedasticity-robust standard errors are consistent because $\hat u_i^2$ (the squared OLS residual) is a ...
3
votes
1answer
37 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
0
votes
0answers
16 views

Source of least square estimator of Poisson parameter

I have $X \sim \text{Poisson}(\theta)$ I need to see how least square estimator of $\theta$ is obtained. Is there anything online showing that?
2
votes
0answers
24 views

If we nonlinearly transform the LS estimates, will they still be unbiased estimates of the true value?

So this is an discussion which came up with a friend/colleague who is a physicist postdoc. He has a bunch of data $(x_i,y_i)$ and wants to fit it to the form $y=e^{ax}$. He uses (weighted) nonlinear ...
1
vote
0answers
30 views

Least Squares Estimation of Poisson Parameter

"Assume independent random variables $Y_i$~$Poisson(λx_i)$. Supposing that $x_i$ are given, fixed constants, obtain the least squares estimator of $λ$ and compute its variance." This kind of a ...
0
votes
0answers
53 views

Should I use stationarity test before OLS regression

I need to know if conducting a stationarity test on the variables, such as the Dickey-Fuller test, is important before doing any regression like OLS? if so, if the variable is stationary after ...
1
vote
0answers
32 views

Solving for GLM coefficients w/ Newton-Raphson

Note : This is a question about a homework problem I am facing. I have some data that is to be modeled with a logistic regression model. I am supposed to do two things: (1) use newton-raphson ...
3
votes
1answer
108 views

Would a simple OLS regression work with this example?

First and foremost I would greatly appreciate any help you can provide me with. I am writing my undergraduate thesis on the rise of populism in France. The relationship I am trying to better ...
1
vote
1answer
30 views

Prove that System FGLS is Consistent

In the Systems of Equations framework, such as Seemingly Unrelated Regression (SUR), suppose we have $g=1,\ldots,G$ equations. Let $\mathbf{X}_i$ be a $G \times K$ matrix, $\mathbf{y}_i$ be $G \times ...
0
votes
1answer
46 views

Is the assumption of linearity necessary for the convergence of the least squares method to the MSE solution

More formally, say there are input vectors $\bf x$ and scalar outputs $Y$ being generated i.i.d. from a joint distribution $p$ and we are interested in estimating $\mu({\bf x}) = {\mathbb ...
1
vote
0answers
30 views

Why sandwich estimators aren't always used in OLS regression?

I asked before what is the intuition behind sandwich estimators. I must still missing something because I don't understand why sandwich estimators are not always applied to OLS residuals. Can you ...
8
votes
1answer
337 views

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
0
votes
2answers
42 views

Condition number of data matrix and stability of OLS estimates

I have a multivariate regression model $Y=X\beta ' + \epsilon$. The variables in the $X$ matrix have very different scales and hence the condition number of $X'X$ is huge (order of trillions). I ...
0
votes
1answer
26 views

How to solve nonlinear optimization problem in R

Suppose I have a set of data and reason to believe the following relation holds y ~ a0 + a1*x1 + a2*x2 + a3*log(x3) How can I use R to solve for the coefficients {a0, ... a3}, supposing I want to ...
5
votes
2answers
219 views

Does adjusting for superfluous variables bias OLS estimates?

The usual textbook treatment of adjusting for superfluous variables in OLS states that the estimator is still unbiased, but may have larger variance (see, for example, Greene, Econometric Analysis, ...
0
votes
1answer
55 views

Which post-hoc is more valid for multiple comparison of an unbalanced lmer-model: lsm or mcp?

After doing a model comparison with my mixed lmer model, I have a model with three main effects, no interaction, say ...
2
votes
1answer
27 views

Least Angle regression coefficient reaches zero after included

In LARS how is it possible that after including a variable it could reach zero again? http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf I understood that it works like: 1) choose ...
0
votes
0answers
41 views

Variance in significance with interaction term

I try to estimate the effects of NatRent (Natural resource rents in % of GDP) on GDP growth per capita (in %). When I include a Rule of Law (a measure for institutional quality) the coefficient of ...
1
vote
0answers
11 views

Help required in controlling for socioeconomic factors when mapping travel behavior

Background I have cleaned disaggregate travel data down to a household level (of about 75% of households) and socio economic data aggregated into 100-200 household blocks. What I'm wanting to do is ...
0
votes
1answer
24 views

Maximum Likelihood solution of a zero-covariance process

Let the measurement model be: $\tilde{y}=Hx+v$ $\tilde{y}=H\hat{x}+e$ where $H$ is the basis matrix, $v$ is a constant vector equal to, say, $a$, $x$ is the measurement variable and $e$ is a ...
1
vote
1answer
39 views

LSmeans - Unbalanced data with interactions

I wish to analyze an unbalanced data set with 3 variables Tleaf, Tair, and orientation (factor with two levels). Considering the effect of the factor "orientation", I wish to determine if "Tair" has a ...
2
votes
1answer
86 views

Estimating standard error of parameters of linear model fitted using gradient descent

Given a linear model $$y = X\beta + \epsilon$$ we can estimate parameters $\hat{\beta}$ using two different ways - ordinary least squares (OLS) and gradient descent (GD). Both of them boil down to ...
0
votes
0answers
10 views

How to compare plsr-selected components (generated from separate regression formulas)?

I would like to identify the subset of landscape variables from one distance class (within a series of different distance classes) which are the best predictors of 'noncol' abundance. There are many ...
3
votes
2answers
83 views

Function Approximation vs. Regression

Some background before I state the questions: I have a $d$-dimensional random vector $X=(X_1,\ldots,X_n)$ and a function $f:\mathbb{R}^d\rightarrow\mathbb{R}$. Ultimately my goal is to understand $f$ ...
18
votes
2answers
524 views

Why does ridge estimate become better than OLS by adding a constant to the diagonal?

I understand that the ridge regression estimate is the $\beta$ that minimizes residual sum of square and a penalty on the size of $\beta$ $$\beta_{ridge} = (\lambda I_D + X'X)^{-1}X'y = ...
0
votes
0answers
103 views

How to derive OLS through MLE? [duplicate]

I am just curious on finding about this derivation
0
votes
0answers
34 views

Two-stage probit least squares

I am estimating a two-stage probit least squares (2SPLS) model. From the readings I have done so far, it appears the first stage of the 2SPLS has to be estimated with a probit, and then a continuous ...
0
votes
2answers
54 views

How to apply a logistic + an OLS model to the same data set?

I have a data set measuring rock detection depths $Y$ based on the distance from some point of interests $X$, which are classified based on geophysical criteria. Each observation $Y$ is set after ...
0
votes
0answers
22 views

Least-squares fitting with only optimum features, after Lasso - valid?

Using Lasso reduces the coefficients of features of a model, reducing some to zero, and thereby performing feature selection. The number of features depends on the value of $\alpha$ aka $\lambda$. In ...
6
votes
1answer
132 views

Proof that the coefficients in an OLS model follow a t-distribution with (n-k) degrees of freedom

Background Suppose we have an Ordinary Least Squares model where we have $k$ coefficients in our regression model, $$\mathbf{y}=\mathbf{X}\mathbf{\beta} + \mathbf{\epsilon}$$ where $\mathbf{\beta}$ ...
1
vote
0answers
18 views

Least Squares Regression - Error [duplicate]

In standard least squares regression, we find constants $\beta_1$ and $\beta_2$ such that the square of the average error, $\epsilon = y_i - (\beta_1 + \beta_2x_i)$, is minimized, and so the 'line of ...
0
votes
0answers
29 views

Help in calculating OLS estimators under certain constraints/modifications

I am new to these forums and to econometrics as a whole. I was hoping someone would be able to give me a nudge in the right direction with this problem. I've done extensive research both online and in ...
2
votes
0answers
57 views

OLS standard error that corrects for autocorrelation but not heteroskedasticity

Question: By mapping the OLS regression into the GMM framework, write the formula for the standard error of the OLS regression coefficients that corrects for autocorrelation but not ...