Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

Filter by
Sorted by
Tagged with
1 vote
1 answer
37 views

How to estimate the coefficients in OLS (all steps) [duplicate]

I'm a B.S. Math graduate who likes to (attempt) to teach myself statistics on my own time because I can't afford a masters degree. It really bothers me when I can't understand at a fundamental level ...
AdmiralMunson's user avatar
1 vote
1 answer
43 views

How does expectation maximization relate to weighted least squares?

For the past few days I have been trying to implement the EM-algorithm in order to segment stores into k-clusters. What I already did was derivation of the complete-log-likelihood and also performed ...
Tim's user avatar
  • 273
1 vote
1 answer
28 views

How to model a standardized index in a regression?

I have a standardized index, $x$ (variations in s.d.) and I want to regress my dependent variable, $y$ on it. In my dataset, the index ranges from approximately -2 to 2, but there is no constraint. ...
Oalvinegro's user avatar
1 vote
1 answer
48 views

Definition of fixed effects used in Journal Papers

I have a question regarding time fixed effects and their definition in empricial Papers. Authors often talk about (1) estimating an OLS Regression and employing time and country fixed effects. What I ...
VivaFalastin's user avatar
1 vote
1 answer
34 views

I didn't understand this part of transposition

Let β = (β0, β1, . . . , βp) be the (p + 1)-vector of coefficients. The vector of predicted values (f (x1), . . . , f (xn))T can be written as Xβ.
AICHA DRAOUI's user avatar
1 vote
1 answer
82 views

rolling mean series - strong autocorrelation

I need to works with time-series of rolling means of a certain variable that comes from the data. I have this rolling means for N different individuals. Call them Y. I would like to test some ...
user9875321__'s user avatar
1 vote
1 answer
76 views

Unbiased and constistency of OLS

For the linear regression $y_t = Bx_t+e_t$ where we have the assumptions: $E(e_t)=0$, $E(e_t^2) = \sigma^2$, $E(e_t e_s)= 0$ for $s\neq t $ ...
WNZ's user avatar
  • 21
1 vote
1 answer
110 views

After selecting variables from lasso regression, is it a good practice to re-run the regression with selected variables?

As the subject suggests, after selecting regressors from lasso regression, is it a good practice to re-run the an ordinary linearly regression with selected variables? I just feel like intuitively, ...
Matt Frank's user avatar
1 vote
2 answers
132 views

Regression with sample split

I have a question with respect to running multiple linear regressions for the entire sample and different subsamples: I have a dataset that includes a dependent variable y and several explanatory ...
derhard's user avatar
  • 11
1 vote
1 answer
82 views

Least Squares Estimation for CRD experiments

Setup Let us use the following model for a Completely Randomised Designed (CRD) experiment: $$ Y_{ij}= \mu + \tau_i + \epsilon_{ij}$$ where the errors $\epsilon _{ij}$ ~ Normal ($0, \sigma ^2 $) and ...
FD_bfa's user avatar
  • 243
1 vote
1 answer
67 views

OLS model and random effects model generate identical outcomes

Using R (package plm), I ran OLS and random effects model to compare the effect of marriage on earnings based on the youth survey longitudinal data. Strangely, both ...
jck21's user avatar
  • 325
1 vote
1 answer
141 views

Gauss Markov Theorem for linear combinations [duplicate]

I know that the Gauss Markov theorem implies that under some conditions, OLS estimates have the smallest variance of all unbiased linear estimators. In particular if I have a model like $ y = \alpha + ...
Lola1993's user avatar
1 vote
1 answer
119 views

What is the difference between standard errors and residuals in OLS?

I'm trying to get a deeper understanding of how OLS works. One thing that I thought I understood is the difference between standard errors and residuals. Here are two definitions Standard errors: The ...
Tomas R's user avatar
  • 177
1 vote
1 answer
26 views

Times series with autocorrelated errors

I'm following the "Time Series Analysis and Its Applications With R Examples" from Shumway and Stoffer. In chapter 3.8 they talk about Regression with Autocorrelated Errors. "They use ...
Warhawk1987's user avatar
1 vote
1 answer
254 views

How to interpret the coefficient of a limited independent Variable (Index)?

I assume this is a very simple question, however I am not sure about it. I have a regression table in front of me that contains the coefficients of a linear regression. The dependent variable is ...
Sam's user avatar
  • 95
1 vote
1 answer
204 views

How to interpret an index of values between -2.5 and +2.5 (an independent variable) in a regression?

I am in the proccess of writing my Master's Thesis and I'm performing a multivariate regression (OLS). One of my independent variables is Chinn-Ito Index (financial openness index) which takes values ...
Victoria's user avatar
1 vote
1 answer
79 views

How exactly is OLS derived from assumption of normally distributed residuals?

Ordinary least square solution of linear regression can be derived from the assumption of normally distributed residuals: $$ e_i=y_i-\hat{y_i}\\ e_i\sim N(0, \sigma^2) $$ What I don't quite understand ...
user2303690's user avatar
1 vote
2 answers
183 views

How is zero conditional mean assumption "stronger" than uncorrelated assumption?

I am trying to understand what the zero conditional mean assumption ($\mathbb{E}[u\vert X]=0 $) encompasses in addition to a zero-correlation assumption ($\text{Corr}(X,u)=0$). I assume it must be &...
phil's user avatar
  • 341
1 vote
1 answer
428 views

Identical OLS and Random Effect Model; Why is my theta/individual effect variance zero?

I have a panel data set with companies and some variables expressed as changes year-to-year from 2017-2018 (2018) to 2020-2021 (2021). I am trying to estimate a regression estimating the effect of all ...
Jonas Esterer's user avatar
1 vote
1 answer
23 views

Reorder dataset to achive LSE between two data sets

Assume I have two datasets, each one containing 5000 samples, and each sample has three dimensions. I am looking for a way to "reorder" the samples in one (or probably both) dataset such ...
Maria's user avatar
  • 43
1 vote
1 answer
476 views

Is it possible to derive the joint probability distribution of squared OLS residuals under the classical linear regression assumptions?

Consider the linear regression model, $$ \boldsymbol{y}=\boldsymbol{X\beta}+\boldsymbol{\epsilon}, $$ where $\boldsymbol{y}$ is an $n$-vector of responses, $\boldsymbol{X}$ is an $n\times p$ matrix of ...
Thomas Farrar's user avatar
1 vote
2 answers
125 views

OLS estimator question: using a subset versus using a dummy-interacted variables

Suppose that we are interested in the following model: $$y_i=\beta_1+\beta_2x_{i2}+\beta_3x_{i3}+u_i$$ Here, there is a dummy variable $d_i$. I am wondering whether the following estimators are ...
M.C. Park's user avatar
  • 905
1 vote
1 answer
106 views

Cointegration test; model with different number of explanatory variables

I have run an ADF test on the residuals of an ARDL and a DOLS model to test for cointegration. I have 3 explanatory variables and 1 response variable. When I run the ADF test on the residuals on both ...
Benjamin Bech's user avatar
1 vote
1 answer
632 views

Different number of observations after including control variables

I have two regression models. I am using paneled data on individuals from 2010 up to 2019. For some individuals, I have several years of observations, whereas for others, there are only 2 or so. The ...
notest's user avatar
  • 41
1 vote
1 answer
56 views

Gradient of the second order term of Newton's Method

I know that Netwon's method can be pushed to the second order using the 1st Taylor expansion. However, how can I generalize Netwon's method to take x_0 as a vector and have the ability take the ...
MadeOfSteelDude's user avatar
1 vote
2 answers
291 views

How to obtain least squares when $X^TX$ cannot be inverted

This work is all theoretical and for school, so we were only provided this information to work with, no actual y values. I have a simple linear model I have been asked to translate into a matrix, ...
barnsm2's user avatar
  • 85
1 vote
1 answer
693 views

How to choose covariates for synthetic control

Trying to construct a synthetic control and I've chosen a set of covariates that are correlated with my outcome variable and significant for P>|t| using OLS. Is choosing covariates for synthetic ...
prismarine's user avatar
1 vote
1 answer
266 views

OLS regression with "categorical" outcome variable (50 categories)

I'd like to do multiple regressions of test scores against a few continuous variables (e.g. age, clinical measures) but the test scores are discrete values (1-10 or 1-50). The function I wanted to ...
RBG's user avatar
  • 35
1 vote
2 answers
204 views

Correlated error restrictions and OLS

I have a VAR model of the form $$ Y_t = \beta Y_{t-1} + \varepsilon_t $$ Where $Y_t$ and $\varepsilon_t$ are $n\times 1$ vectors, and $\beta$ is an $n \times n$ matrix. The residuals $\varepsilon_{t,i}...
Mich55's user avatar
  • 117
1 vote
1 answer
184 views

Does OLS bias Sigma estimate when residuals are non-normal compared to MLE estimate?

I've come across a textbook problem from, Gelman - Regression and Other Stories that asks to compare MLE parameter values on a small dataset where n = 16 ...
McGez's user avatar
  • 91
1 vote
1 answer
256 views

Show unbiased OLS estimator and expression for variance of OLS estimator

Consider the usual linear mixed model: $$Y=X \beta+ZB+\epsilon $$ where Y and $\epsilon$ are $n$-dimensional random variables and $B$ is a $q$-dimensional random variable independent of $\epsilon$ so ...
Lifeni's user avatar
  • 305
1 vote
1 answer
147 views

Properties of OLS estimator for overlapping observations' regression

I am running regressions of long-horizon gross financial returns $$R_{t+k} = \prod_{i=1}^{k} R_{t+i}$$ where $R_t=1+r_t$, on current dividend-price ratios $DP_t=\frac{D}{P}_t$: $$R_{t+k}=\alpha +\beta ...
Mr Frog's user avatar
  • 339
1 vote
1 answer
568 views

Can $R^2$ be applied to non-linear least square regression? [duplicate]

$R^2$ is usually used as a measure to determine a goodness of a fit. It appears to be used often times for linear least square fits, linear regression. There's another measure which is RSS (residual ...
bonCodigo's user avatar
  • 131
1 vote
1 answer
72 views

Why is multiple R squared artificially inflated?

Why is multiple $R^2$ artificially inflated? I understand that the value of $R^2$ cannot decrease with additional variables, but why does it increase when an additional variable is added that has no ...
Brigadeiro's user avatar
1 vote
1 answer
317 views

Is there a simple generalization of the OLS "weighted average of the slopes" interpretation of the slope coefficient to multivariate predictors?

The slope coefficient of univariate (single predictor, single response) OLS can be written as a weighted average of the pairwise slopes between $(x,y)$ pairs, specifically: $$ \hat{\beta} = \frac{\...
Bob Durrant's user avatar
1 vote
1 answer
31 views

What type of regression/estimation technique is suitable?

I am modelling the dynamic conditional correlations of a couple of assets via DCC mgarch. I also have some exogenous variables that try to explain these correlations. Since my dependent variable is ...
Abderrahim's user avatar
1 vote
1 answer
271 views

Reducing a linear regression (OLS) model by dropping non-significant coefficients

Would it be proper for me to reduce a model by iterating though the coefficients and dropping the ones with high p-values and then refitting and doing this again until all coefficients are significant?...
s5s's user avatar
  • 675
1 vote
1 answer
121 views

How to set a constraint for a non-linear least squares problem [closed]

I am trying to fit some data where the cost function is $ax^2 + bx + c$ and I need to have $a+b+c = 1$. How do I set such a constraint in MATLAB or Python?
SEU's user avatar
  • 111
1 vote
1 answer
404 views

Why strict exogeneity implies that the regressor is uncorrelated with the error term?

One of the assumptions of the classical linear regression model is strict exogeneity, that is: $ E[\epsilon | X ] = 0 $ This should imply that the regressors are uncorrelated with the errors: $E[X^T \...
robertspierre's user avatar
1 vote
1 answer
1k views

ARIMA fit vs. Linear Regression -- which one to use?

I have a data set made up of 30 observations, years 1980-2020. The second variable is the amount of cargo passing through a particular harbor on the west coast (in tons). I have attempted to fit an ...
amvoight's user avatar
1 vote
1 answer
488 views

Is Least Squares estimator for linear model the unique minimum variance unbiased estimator for a linear model?

I am following Linear Modfels in Statistics, Rencher & Schaalje, 2nd Edicition for the proof of Gauss-Markov Theorem (Theorem 7.3d, Page 146). I understand how least squares estimator $\mathbf{\...
ethelion's user avatar
1 vote
1 answer
74 views

Ordinary least squares estimation explained in "Optimal Design of Experiments" by Goos/Jones, questions about some equationsEq

I am currently trying to become familiar with design of experiments with the book “Optimal design of experiments” by Goos and Jones. In chapter 2, they discuss the use of a Plackett–Burman type design ...
Soltub's user avatar
  • 11
1 vote
1 answer
220 views

Curve fitting - multiple indepedent fits or a single combined fit?

Assume, I have a list of actual, noisy, independent scalar measurements [y1, y2, ..., yn] (think: y1 = [0%, 12%, 52%, 79%, 99%]) for a scalar series ...
Mr. T's user avatar
  • 111
1 vote
1 answer
34 views

Transformation in regression with no regressors and equi-correlated disturbances

I'm working on a problem in which we consider a simple regression with no regressors and equi-correlated disturbances. So we have $y_i = \alpha + \varepsilon_i$ where $E[\varepsilon_i, \varepsilon_j] =...
Phil's user avatar
  • 109
1 vote
1 answer
92 views

Subtracting a constant from the OLS summation

On a proof for the OLS of $\beta$, I have seen this step: $\sum x_i (y_i - \alpha - \beta x_i) = \sum (x_i - k) (y_i - \alpha - \beta x_i) $ for any constant $k$. Why is this true?
Sparsity's user avatar
  • 584
1 vote
1 answer
245 views

Existence of least squares and maximum likelihood estimators?

In statistical parameter estimation where there is a deterministic and stochastic component to the observation-generating model, do least squares and maximum likelihood estimators always exist? ...
hatmatrix's user avatar
  • 859
1 vote
1 answer
133 views

What is the correct way to write the model equation for a linear probability model?

I'm trying to write down the equation describing a linear probability model. If I was writing out the equation for an OLS model with continuous y with observation unit i , I would write: $$y_i = \...
WildGunman's user avatar
1 vote
2 answers
70 views

If I add an extra variable to a regression model I already have, and the R squared increases, does this usually mean the new model is better?

Lets say I'm estimating a model with ordinary least squares, and that my initial model (with an $R^2$ of 0.5) is $$ y_i = \beta_o + \beta_1x_{i1} + \beta_2x_{i2} + \varepsilon_i $$ Lets say I add a ...
ElBuenMarvin's user avatar
1 vote
1 answer
507 views

How to derive the covariance matrix between $\bar{y}$ and $\hat{\beta_c}$ where $\hat{\beta_c}$ is the OLS estimator of a linear model?

$$cov\begin{bmatrix}\bar{y}\\\hat{\beta}_c\end{bmatrix}=\sigma^2\begin{bmatrix}\frac{1}{n} & 0^T\\0 & (X_c^TX_c)^{-1}\end{bmatrix}$$ with $\hat{\beta}_c=(X_c^TX)^{-1}X^T_cy$. I am supposed to ...
JoZ's user avatar
  • 679
1 vote
3 answers
2k views

Endogeneity testing using correlation test

I am currently testing my linear model using OLS method. The last thing I have to test is endogeneity issue. Is it enough if I test each explanatory variable for correletion with error term? Than ...
sabiste's user avatar
  • 21

1
30 31
32
33 34
57