Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

learn more… | top users | synonyms (1)

2
votes
1answer
39 views

numerical difference between sum of squared residuals and likelihood

I previously asked a question that got labelled as duplicated because I did not explain it correctly. I should not have used the regression model as an example because I can see how, by using that as ...
0
votes
0answers
14 views

Least squares: Calculus to find residual minimizers?

Reading a section on simple regression in "An Introduction to Statistical Learning with Applications in R" I got a question on residual sum of squares minimization. Quoting from the book: ... ...
2
votes
1answer
69 views

Why is $E[u] = 0$ in OLS and what is the difference between the error and constant?

How is the constant term in a linear least squares regression different from the error term? Why does the error term have a normal distribution with mean "0" and sd $\sigma$? I mean I can see that ...
0
votes
2answers
91 views

Residual vs. fits

Based on the plot, would it be OK to assume the errors have mean zero (aprox. half of them are under and the other half above zero line) given the strict exogeneity assumption by OLS?
1
vote
0answers
56 views

Bayes Linear regression- logarithmic transformation of prior distribution of the variance

I have a Bayesian version of a linear regression with 3 covariates. The model is given by \begin{align*} Y\sim N(\mu,\tau)\end{align*} \begin{align*} \mu=\alpha + \sum\beta_{i}x_{i}\end{align*} where ...
1
vote
0answers
24 views

Different values predicted by OLS model and Time Series model

Let us say , I have an explanatory variable X and a dependent variable Y and I use OLS and find Y = 0.5 + 3X. Now let us assume that both X and Y are time series data, so using ARIMA modelling ...
1
vote
0answers
36 views

Equivalence of the partial least square regresssion's iterative algorithm and its optimization problem

I am reading The Elements of Statistical Learning. This is a page from the partial least square section: The exercise asks to prove the equivalence between Algorithm 3.3 and Eq. (3.64). Here's my ...
0
votes
1answer
20 views

Is testing for endogeneity necessary?

I was wondering if testing for endogeneity is necessary when doing an OLS multiple regression. Similarly, is testing for autocorrelation necessary if there is no time series data? Is testing for ...
0
votes
0answers
39 views

Least squares method for parameter estimation in AR(1) model

In order to estimate parameters $ \mu $ and $ \alpha$ least squares method can be used.This is what I did to find the least squares. $S=\sum_{t=2}^n(X_t-\mu-\alpha X_{t-1}+\alpha\mu)^2$. And ...
0
votes
1answer
29 views

How to explain change of sign on regression coefficient when another variable is added to OLS model? [duplicate]

I am trying to run an OLS regression, with log of per capita calorie as my dependent variable and age and years of education of household head, log per capita expenditure as my independent variables ...
3
votes
0answers
17 views

Will a pooled coefficient be bounded by the coefficients from the group regressions? [duplicate]

Consider the linear regression model, $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + u, u \sim N(0, \sigma^2 )$ If we estimate the model twice using OLS on two mutually exclusive groups (say, group 1 and ...
0
votes
1answer
70 views

Linear model vs. linear regression

I have a question that I find really confusing regarding linear modelling and linear regression. I have expectation regarding the way some dependent variable (DV) are going to evolve with an ...
1
vote
1answer
26 views

Statistical model that finds a coordinate (x,y) that minimizes the distance from a group of coordinates

A example would be if I launched a 100 tennis balls in the air and plotted the coordinates of where each landed. I would like to be able to find the point in the center of all those coordinates. I ...
0
votes
0answers
15 views

Does forward filtering result in OLS estimate for each time point?

I'm learning about dynamic linear models and was trying to think about the relationship between GLS and forward filtering (Kalman filtering where the state is the vector of parameters). Here's my ...
1
vote
1answer
55 views

Added interaction term, standard errors inflated

I am running a simple regression of an index of cardiovascular health (Heart Rate Variability) on Age and Gender (as a dummy variable), n=430. I first ran: $$HRV \sim \beta_0 + \beta_1Age ...
0
votes
1answer
25 views

Determine weights in weighted least squares regression

Assume we have a cross-section of $N$ stocks. $Y_i$ is an sample variance estimate of stock returns for stock $i$. This sample variance is estimated using $T_i$ number of observations. All $T_i$ are ...
1
vote
1answer
23 views

Covariance of OLS estimator and residual = 0. Where is the mistake?

$Cov(b,e|X)$, where $b$ is the OLS estimator of the coefficients, $e$ is the residual vector, and $X$ is the regressor matrix. We know that $Cov(b,e|X)=E(be'|X)-E(b|X)E(e'|X)$ where ' $'$ ' is the ...
0
votes
0answers
27 views

OLS with dummy and few observations (12 mean values from 2.5k observations)

I have a dataset of 12 markets, in total ~2.5k trades happened over all markets. I now calculated 6 different measures for the each markets performance (my 12 observations per measure I want to use as ...
0
votes
0answers
19 views

Asymptotic distribution of t-ratio

I am looking at a problem where I have to calculate the asymptotic distribution of a t-ratio, after having run a OLS regression. I have re-written the expression so as to be t = z * sqrt ...
0
votes
1answer
60 views

About stepwise regression and correlation

I am trying to fit a model to some observed data with the least squares method. Now, I am at the stage where I have run a stepwise regression (traditional), with Entry level $=0.025$ and Stay level ...
8
votes
2answers
158 views

How do residuals relate to the underlying disturbances?

In the least squares method we want to estimate the unknown parameters in the model: $$Y_j = \alpha + \beta x_j + \varepsilon_j \enspace (j=1...n)$$ Once we have done that (for some observed ...
0
votes
0answers
13 views

Trouble with horner function in MATLAB [closed]

I have the following homework question: Apply linear least squares with the two models S1(A, B, C) = Ax^2 + Bx + C and S2(A, B, C, D) = Ax^3 + Bx^2 + Cx + D to the data set (0, 4), (1, −1), (2, ...
0
votes
0answers
29 views

OLS derivation question [duplicate]

How come I always see the derivation of $\hat{\beta}$ in OLS using matrix differentiation and solving for when the derivative is $0$. Couldn't one just derive it also by noting that in $Y = X\beta + ...
1
vote
1answer
38 views

Why does this have zero sample correlation?

http://web.stanford.edu/~mrosenfe/soc_meth_proj3/matrix_OLS_NYU_notes.pdf On page 4, it says that $x_k'e = 0$ implies that each regressor has zero sample correlation with the residuals. I don't see ...
0
votes
0answers
21 views

creating an indexed dummy variable as a predictor in OLS

I am performing on OLS with two predictors and a response variable. The data is a time series of 450 days approximately. There is an irregular pattern in my response variable - it sometimes ...
2
votes
0answers
28 views

How is the use of OLS in estimating ARCH(q) models justified?

So when estimating ARCH(q) model, some procedures go like estimating first AR(q) model using OLS and then using OLS again against error terms of AR(q). But ARCH suggests heteroskedasticity, and OLS ...
1
vote
1answer
35 views

Hypothesis test for the response variable in a least squares regression model

I have an equation where time it takes to get to work is based on time it takes to depart, number of red lights hit, and number of trains you encounter. The model is shown below: ...
0
votes
1answer
18 views

Comparing how good models with binary dependent variable are

So I'm not very well versed when it comes to statistics, so this is a bit over my head. I can't go into too much detail (NDA, etc etc), but I have four different datasets where in every dataset there ...
0
votes
1answer
73 views

Weighted least-squares negative fitted values

I am running a weighted least-squares regression (where all weights are strictly positive), where my dependent variable is a cross-section of variance values. Since variance is always positive (>=0), ...
4
votes
2answers
158 views

How to know if “best fit line” really represents known set of data?

I have a known set of data. I have created a "linear best fit line" for that set of data. Is there a way to determine how well my set of data fit that best fit line (some sort of score)? I'm very ...
0
votes
0answers
9 views

Performing OLS with gamma transformation

In some specific areas it is common to perform OLS regresion with beta distribution transformation. The α and b parameters are calculated by the sample's μ and σ^2. While the transformed dependent ...
1
vote
1answer
41 views

Weighted Least Squares Through The Origin

For a regression model through the origin, with Var($e_i|x_i) = x_i^2σ^2$ . The corresponding regression model is $Y_i$ = $\beta$$X_i$ +$e_i$. How do I create a least squares model? I know I need to ...
0
votes
0answers
18 views

OLS and multiple hypothesis testing

I have a multivariate regression with more than two independent variables. However, I am only interested in two particular variables. So for simplicity, I write this model in terms of the two ...
3
votes
1answer
32 views

Expected value of least squares estimator $\hat{\beta}$

Given $\hat{\beta} = (X^{T}X)^{-1}(X^{T}Y)$, how do you derive the expected value? I found answers for finding the variance matrix but not the expected value. Thank you kindly.
2
votes
0answers
29 views

Aggregate variance for ordinary least squares?

I am writing some MapReduce code to calculate ordinary least squares from a sample of data. I'd like to include standard error, but I am running into a problem in calculating the variance of the ...
1
vote
1answer
34 views

Consistency of heteroskedasticity-robust standard errors

Is the following statement true or not? When applying OLS with model $y=a+bx+u$, the heteroskedasticity-robust standard errors are consistent because $\hat u_i^2$ (the squared OLS residual) is a ...
3
votes
1answer
40 views

Inference about the true intercept of the model and the OLS being BLUE

Consider the following population regression model: $$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$ where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = ...
0
votes
0answers
16 views

Source of least square estimator of Poisson parameter

I have $X \sim \text{Poisson}(\theta)$ I need to see how least square estimator of $\theta$ is obtained. Is there anything online showing that?
2
votes
0answers
26 views

If we nonlinearly transform the LS estimates, will they still be unbiased estimates of the true value?

So this is an discussion which came up with a friend/colleague who is a physicist postdoc. He has a bunch of data $(x_i,y_i)$ and wants to fit it to the form $y=e^{ax}$. He uses (weighted) nonlinear ...
1
vote
0answers
38 views

Least Squares Estimation of Poisson Parameter

"Assume independent random variables $Y_i$~$Poisson(λx_i)$. Supposing that $x_i$ are given, fixed constants, obtain the least squares estimator of $λ$ and compute its variance." This kind of a ...
0
votes
0answers
59 views

Should I use stationarity test before OLS regression

I need to know if conducting a stationarity test on the variables, such as the Dickey-Fuller test, is important before doing any regression like OLS? if so, if the variable is stationary after ...
1
vote
0answers
36 views

Solving for GLM coefficients w/ Newton-Raphson

Note : This is a question about a homework problem I am facing. I have some data that is to be modeled with a logistic regression model. I am supposed to do two things: (1) use newton-raphson ...
3
votes
1answer
121 views

Would a simple OLS regression work with this example?

First and foremost I would greatly appreciate any help you can provide me with. I am writing my undergraduate thesis on the rise of populism in France. The relationship I am trying to better ...
1
vote
1answer
36 views

Prove that System FGLS is Consistent

In the Systems of Equations framework, such as Seemingly Unrelated Regression (SUR), suppose we have $g=1,\ldots,G$ equations. Let $\mathbf{X}_i$ be a $G \times K$ matrix, $\mathbf{y}_i$ be $G \times ...
0
votes
1answer
49 views

Is the assumption of linearity necessary for the convergence of the least squares method to the MSE solution

More formally, say there are input vectors $\bf x$ and scalar outputs $Y$ being generated i.i.d. from a joint distribution $p$ and we are interested in estimating $\mu({\bf x}) = {\mathbb ...
1
vote
0answers
34 views

Why sandwich estimators aren't always used in OLS regression?

I asked before what is the intuition behind sandwich estimators. I must still missing something because I don't understand why sandwich estimators are not always applied to OLS residuals. Can you ...
8
votes
1answer
362 views

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
0
votes
2answers
53 views

Condition number of data matrix and stability of OLS estimates

I have a multivariate regression model $Y=X\beta ' + \epsilon$. The variables in the $X$ matrix have very different scales and hence the condition number of $X'X$ is huge (order of trillions). I ...
0
votes
1answer
32 views

How to solve nonlinear optimization problem in R

Suppose I have a set of data and reason to believe the following relation holds y ~ a0 + a1*x1 + a2*x2 + a3*log(x3) How can I use R to solve for the coefficients {a0, ... a3}, supposing I want to ...
5
votes
2answers
229 views

Does adjusting for superfluous variables bias OLS estimates?

The usual textbook treatment of adjusting for superfluous variables in OLS states that the estimator is still unbiased, but may have larger variance (see, for example, Greene, Econometric Analysis, ...