Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

0
votes
1answer
20 views

How to explain change of sign on regression coefficient when another variable is added to OLS model? [duplicate]

I am trying to run an OLS regression, with log of per capita calorie as my dependent variable and age and years of education of household head, log per capita expenditure as my independent variables ...
1
vote
0answers
16 views

MATLAB curve fitting - least squares method - wrong “fit” using high degrees [migrated]

Anyone here that could help me with the following problem? The following code calculates the best polynomial fit to a given data-set, that is; a polynomial of a specified degree. Unfortunately, ...
3
votes
0answers
17 views

Will a pooled coefficient be bounded by the coefficients from the group regressions? [duplicate]

Consider the linear regression model, $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + u, u \sim N(0, \sigma^2 )$ If we estimate the model twice using OLS on two mutually exclusive groups (say, group 1 and ...
0
votes
1answer
53 views

Linear model vs. linear regression

I have a question that I find really confusing regarding linear modelling and linear regression. I have expectation regarding the way some dependent variable (DV) are going to evolve with an ...
1
vote
1answer
26 views

Statistical model that finds a coordinate (x,y) that minimizes the distance from a group of coordinates

A example would be if I launched a 100 tennis balls in the air and plotted the coordinates of where each landed. I would like to be able to find the point in the center of all those coordinates. I ...
0
votes
0answers
9 views

Obtain the Beta of a regression using MLE Matlab

I am trying to find an alternative way around OLS to find the Beta coefficients so that they give me $\epsilon$ My depedent variable (y) is a time series, and my independent variable (x) is just a ...
0
votes
0answers
15 views

Does forward filtering result in OLS estimate for each time point?

I'm learning about dynamic linear models and was trying to think about the relationship between GLS and forward filtering (Kalman filtering where the state is the vector of parameters). Here's my ...
1
vote
1answer
50 views

Added interaction term, standard errors inflated

I am running a simple regression of an index of cardiovascular health (Heart Rate Variability) on Age and Gender (as a dummy variable), n=430. I first ran: $$HRV \sim \beta_0 + \beta_1Age ...
0
votes
1answer
18 views

Determine weights in weighted least squares regression

Assume we have a cross-section of $N$ stocks. $Y_i$ is an sample variance estimate of stock returns for stock $i$. This sample variance is estimated using $T_i$ number of observations. All $T_i$ are ...
1
vote
1answer
23 views

Covariance of OLS estimator and residual = 0. Where is the mistake?

$Cov(b,e|X)$, where $b$ is the OLS estimator of the coefficients, $e$ is the residual vector, and $X$ is the regressor matrix. We know that $Cov(b,e|X)=E(be'|X)-E(b|X)E(e'|X)$ where ' $'$ ' is the ...
0
votes
0answers
23 views

OLS with dummy and few observations (12 mean values from 2.5k observations)

I have a dataset of 12 markets, in total ~2.5k trades happened over all markets. I now calculated 6 different measures for the each markets performance (my 12 observations per measure I want to use as ...
0
votes
0answers
17 views

Asymptotic distribution of t-ratio

I am looking at a problem where I have to calculate the asymptotic distribution of a t-ratio, after having run a OLS regression. I have re-written the expression so as to be t = z * sqrt ...
0
votes
1answer
57 views

About stepwise regression and correlation

I am trying to fit a model to some observed data with the least squares method. Now, I am at the stage where I have run a stepwise regression (traditional), with Entry level $=0.025$ and Stay level ...
8
votes
2answers
155 views

How do residuals relate to the underlying disturbances?

In the least squares method we want to estimate the unknown parameters in the model: $$Y_j = \alpha + \beta x_j + \varepsilon_j \enspace (j=1...n)$$ Once we have done that (for some observed ...
0
votes
0answers
10 views

Trouble with horner function in MATLAB [closed]

I have the following homework question: Apply linear least squares with the two models S1(A, B, C) = Ax^2 + Bx + C and S2(A, B, C, D) = Ax^3 + Bx^2 + Cx + D to the data set (0, 4), (1, −1), (2, ...
0
votes
0answers
29 views

OLS derivation question [duplicate]

How come I always see the derivation of $\hat{\beta}$ in OLS using matrix differentiation and solving for when the derivative is $0$. Couldn't one just derive it also by noting that in $Y = X\beta + ...
1
vote
1answer
36 views

Why does this have zero sample correlation?

http://web.stanford.edu/~mrosenfe/soc_meth_proj3/matrix_OLS_NYU_notes.pdf On page 4, it says that $x_k'e = 0$ implies that each regressor has zero sample correlation with the residuals. I don't see ...
0
votes
0answers
19 views

creating an indexed dummy variable as a predictor in OLS

I am performing on OLS with two predictors and a response variable. The data is a time series of 450 days approximately. There is an irregular pattern in my response variable - it sometimes ...
2
votes
0answers
22 views

How is the use of OLS in estimating ARCH(q) models justified?

So when estimating ARCH(q) model, some procedures go like estimating first AR(q) model using OLS and then using OLS again against error terms of AR(q). But ARCH suggests heteroskedasticity, and OLS ...
1
vote
1answer
35 views

Hypothesis test for the response variable in a least squares regression model

I have an equation where time it takes to get to work is based on time it takes to depart, number of red lights hit, and number of trains you encounter. The model is shown below: ...
0
votes
1answer
17 views

Comparing how good models with binary dependent variable are

So I'm not very well versed when it comes to statistics, so this is a bit over my head. I can't go into too much detail (NDA, etc etc), but I have four different datasets where in every dataset there ...
0
votes
1answer
69 views

Weighted least-squares negative fitted values

I am running a weighted least-squares regression (where all weights are strictly positive), where my dependent variable is a cross-section of variance values. Since variance is always positive (>=0), ...
4
votes
2answers
133 views

How to know if “best fit line” really represents known set of data?

I have a known set of data. I have created a "linear best fit line" for that set of data. Is there a way to determine how well my set of data fit that best fit line (some sort of score)? I'm very ...
0
votes
0answers
8 views

Performing OLS with gamma transformation

In some specific areas it is common to perform OLS regresion with beta distribution transformation. The α and b parameters are calculated by the sample's μ and σ^2. While the transformed dependent ...
1
vote
1answer
37 views

Weighted Least Squares Through The Origin

For a regression model through the origin, with Var($e_i|x_i) = x_i^2σ^2$ . The corresponding regression model is $Y_i$ = $\beta$$X_i$ +$e_i$. How do I create a least squares model? I know I need to ...
0
votes
0answers
17 views

OLS and multiple hypothesis testing

I have a multivariate regression with more than two independent variables. However, I am only interested in two particular variables. So for simplicity, I write this model in terms of the two ...
3
votes
1answer
27 views

Expected value of least squares estimator $\hat{\beta}$

Given $\hat{\beta} = (X^{T}X)^{-1}(X^{T}Y)$, how do you derive the expected value? I found answers for finding the variance matrix but not the expected value. Thank you kindly.
2
votes
0answers
29 views

Aggregate variance for ordinary least squares?

I am writing some MapReduce code to calculate ordinary least squares from a sample of data. I'd like to include standard error, but I am running into a problem in calculating the variance of the ...
1
vote
1answer
33 views

Consistency of heteroskedasticity-robust standard errors

Is the following statement true or not? When applying OLS with model $y=a+bx+u$, the heteroskedasticity-robust standard errors are consistent because $\hat u_i^2$ (the squared OLS residual) is a ...
3
votes
1answer
39 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
0
votes
0answers
16 views

Source of least square estimator of Poisson parameter

I have $X \sim \text{Poisson}(\theta)$ I need to see how least square estimator of $\theta$ is obtained. Is there anything online showing that?
2
votes
0answers
26 views

If we nonlinearly transform the LS estimates, will they still be unbiased estimates of the true value?

So this is an discussion which came up with a friend/colleague who is a physicist postdoc. He has a bunch of data $(x_i,y_i)$ and wants to fit it to the form $y=e^{ax}$. He uses (weighted) nonlinear ...
1
vote
0answers
32 views

Least Squares Estimation of Poisson Parameter

"Assume independent random variables $Y_i$~$Poisson(λx_i)$. Supposing that $x_i$ are given, fixed constants, obtain the least squares estimator of $λ$ and compute its variance." This kind of a ...
0
votes
0answers
56 views

Should I use stationarity test before OLS regression

I need to know if conducting a stationarity test on the variables, such as the Dickey-Fuller test, is important before doing any regression like OLS? if so, if the variable is stationary after ...
1
vote
0answers
35 views

Solving for GLM coefficients w/ Newton-Raphson

Note : This is a question about a homework problem I am facing. I have some data that is to be modeled with a logistic regression model. I am supposed to do two things: (1) use newton-raphson ...
3
votes
1answer
116 views

Would a simple OLS regression work with this example?

First and foremost I would greatly appreciate any help you can provide me with. I am writing my undergraduate thesis on the rise of populism in France. The relationship I am trying to better ...
1
vote
1answer
33 views

Prove that System FGLS is Consistent

In the Systems of Equations framework, such as Seemingly Unrelated Regression (SUR), suppose we have $g=1,\ldots,G$ equations. Let $\mathbf{X}_i$ be a $G \times K$ matrix, $\mathbf{y}_i$ be $G \times ...
0
votes
1answer
48 views

Is the assumption of linearity necessary for the convergence of the least squares method to the MSE solution

More formally, say there are input vectors $\bf x$ and scalar outputs $Y$ being generated i.i.d. from a joint distribution $p$ and we are interested in estimating $\mu({\bf x}) = {\mathbb ...
1
vote
0answers
32 views

Why sandwich estimators aren't always used in OLS regression?

I asked before what is the intuition behind sandwich estimators. I must still missing something because I don't understand why sandwich estimators are not always applied to OLS residuals. Can you ...
8
votes
1answer
349 views

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
0
votes
2answers
44 views

Condition number of data matrix and stability of OLS estimates

I have a multivariate regression model $Y=X\beta ' + \epsilon$. The variables in the $X$ matrix have very different scales and hence the condition number of $X'X$ is huge (order of trillions). I ...
0
votes
1answer
31 views

How to solve nonlinear optimization problem in R

Suppose I have a set of data and reason to believe the following relation holds y ~ a0 + a1*x1 + a2*x2 + a3*log(x3) How can I use R to solve for the coefficients {a0, ... a3}, supposing I want to ...
5
votes
2answers
228 views

Does adjusting for superfluous variables bias OLS estimates?

The usual textbook treatment of adjusting for superfluous variables in OLS states that the estimator is still unbiased, but may have larger variance (see, for example, Greene, Econometric Analysis, ...
0
votes
1answer
72 views

Which post-hoc is more valid for multiple comparison of an unbalanced lmer-model: lsm or mcp?

After doing a model comparison with my mixed lmer model, I have a model with three main effects, no interaction, say ...
2
votes
1answer
27 views

Least Angle regression coefficient reaches zero after included

In LARS how is it possible that after including a variable it could reach zero again? http://www.cc.gatech.edu/~isbell/reading/papers/lasso_simple.html.pdf I understood that it works like: 1) choose ...
0
votes
0answers
42 views

Variance in significance with interaction term

I try to estimate the effects of NatRent (Natural resource rents in % of GDP) on GDP growth per capita (in %). When I include a Rule of Law (a measure for institutional quality) the coefficient of ...
1
vote
0answers
13 views

Help required in controlling for socioeconomic factors when mapping travel behavior

Background I have cleaned disaggregate travel data down to a household level (of about 75% of households) and socio economic data aggregated into 100-200 household blocks. What I'm wanting to do is ...
0
votes
1answer
24 views

Maximum Likelihood solution of a zero-covariance process

Let the measurement model be: $\tilde{y}=Hx+v$ $\tilde{y}=H\hat{x}+e$ where $H$ is the basis matrix, $v$ is a constant vector equal to, say, $a$, $x$ is the measurement variable and $e$ is a ...
1
vote
1answer
41 views

LSmeans - Unbalanced data with interactions

I wish to analyze an unbalanced data set with 3 variables Tleaf, Tair, and orientation (factor with two levels). Considering the effect of the factor "orientation", I wish to determine if "Tair" has a ...
2
votes
1answer
91 views

Estimating standard error of parameters of linear model fitted using gradient descent

Given a linear model $$y = X\beta + \epsilon$$ we can estimate parameters $\hat{\beta}$ using two different ways - ordinary least squares (OLS) and gradient descent (GD). Both of them boil down to ...