Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

implement i.year i.id FE from stata in python [closed]

When I have to control for fixed effects (time and id) in Stata, I usually run the regression with i.year and i.id (these are my var names). Now, I have to use python for a different project because ...
user avatar
0 votes
0 answers
12 views

The Variance Covariance Matrix of an Estimator Stacking Two OLS Estimators

I am looking for how to derive the variance covariance matrix (henceforth, VCOV) of an estimator stacking two OLS estimators. Suppose that we have two OLS estimators: $$\hat{\alpha}\sim N(\alpha,\;\...
user avatar
  • 501
1 vote
0 answers
19 views

When does the Instrumental Variable (IV) method fail?

Consider the following ARMAX model: $$ y[k]=\alpha y[k-1]+\beta u[k-1]+e[k]-0.7e[k-1] $$ where $e[k]$ is a white noise. One can see that $e[k]-0.7e[k-1]$ is a filtered noise and thus using ordinary ...
user avatar
  • 111
1 vote
1 answer
42 views

Is it possible to derive the joint probability distribution of squared OLS residuals under the classical linear regression assumptions?

Consider the linear regression model, $$ \boldsymbol{y}=\boldsymbol{X\beta}+\boldsymbol{\epsilon}, $$ where $\boldsymbol{y}$ is an $n$-vector of responses, $\boldsymbol{X}$ is an $n\times p$ matrix of ...
user avatar
0 votes
0 answers
21 views

Linear regression with multiple observations for each input point: WLS approach

Consider a dataset $X, y$ where each $y$ is distinct, but many of the rows of $X$ are repeated. Suppose that there are indices $\{ i_j: j = 1 \dots J \}$ such that the consecutive rows $X_{i_j}=X_{...
user avatar
0 votes
0 answers
6 views

Why include fixed effect in cross sectional ols?

Consider repeated cross-sectional data that contains observations in 10 provinces from 2011 to 2015. Each observation is an individual, but the same person is not observed each year, instead, ...
user avatar
  • 1
1 vote
2 answers
35 views

OLS estimator question: using a subset versus using a dummy-interacted variables

Suppose that we are interested in the following model: $$y_i=\beta_1+\beta_2x_{i2}+\beta_3x_{i3}+u_i$$ Here, there is a dummy variable $d_i$. I am wondering whether the following estimators are ...
user avatar
  • 501
0 votes
0 answers
17 views

Scale location plot interpretation

I ran a regular OLS regression and wanted to check if the assumptions for OLS regression was meet. To do this I plotted a scale location plot, but I'm struggeling with the interpretation of the result....
user avatar
  • 1
1 vote
1 answer
27 views

Inclusion of year and seasons as variable for regression with non-stationary response?

The common knowledge is that OLS only makes sense if both the response and explanatory variables are stationary (ignoring exceptions like cointegration), as otherwise, there could be effects of ...
user avatar
  • 101
2 votes
0 answers
36 views

What are some well-known unbiased estimator of regression coefficient besides OLS estimator?

Is there any other unbiased estimator of regression coefficient than OLS? For instance, one might consider using unbiased estimator with less computational cost (since OLS involves matrix inversion)?
user avatar
0 votes
0 answers
9 views

Unbalanced Sample and interpreting log-transformation

I am analyzing a sample of two groups of 5,092 and 114,038 customers - and two very basic questions I assume. Question 1: Do you think this is a problem in general AND for the usage of OLS, fixed-...
user avatar
  • 11
0 votes
0 answers
16 views

Testing equality of coefficients from two different samples

I have the regression statistics for the same regression run on two different samples, and am asked to explain whether it is possible to test for equality of the coefficents, $\beta_1$and $\beta_2$ ...
user avatar
0 votes
1 answer
23 views

What is the intercept in a regression model with demeaned dependent variable?

Suppose you have a regression model $\tilde{y}$ = $X\beta$ + $\varepsilon$, where $\tilde{y}$ = $y$ - $\bar{y}$ and $X$ contains a constant. If you estimate the model by OLS, does the estimated ...
user avatar
0 votes
0 answers
27 views

Discrete "Jumps" in Independent Variable

Suppose I have an independent variable which is available on a monthly basis as follows - the variable assumes a certain value for a given quarter, and then changes in the following quarter. Therefore,...
user avatar
  • 243
0 votes
0 answers
26 views

Covariance matrix of errors for homoskedasticity/heteroskedasticity

I've seen homoskedasticty and heteroskedasticity defined as the following The error term of our regression model is homoskedastic if the variance of the conditional distribution of $u_{i}$ given $X_{...
user avatar
  • 93
1 vote
1 answer
16 views

Can very large count outcomes be treated as continuous variables?

I have very little expertise with count outcomes and analysis of them, but I understand that, in general, they cannot be treated as continuous dependent variables for the purpose of analysis due to ...
user avatar
1 vote
0 answers
30 views

Intuition relating OLS Variance-Covariance matrix and the OLS sampling variance equation

I'm looking at the variances of OLS slope estimators and I found the following equation that holds under the Gauss-Markov assumptions. Suppose that we have a multiple regression model of the following ...
user avatar
1 vote
1 answer
27 views

Does an endogenous variable bias the coefficient of the exogenous one?

We have the following model: $$ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon. $$ We know that: \begin{align*} \operatorname{Cov}(x_1, \epsilon) &\neq 0 \\ \operatorname{Cov}(x_2, \epsilon) &...
user avatar
0 votes
0 answers
13 views

What is the meaning of the inner product between two regression variables?

I have been analyzing the effect of design matrix columns on the contour line of the least squares regression. These contours obviously are ellipses when only two columns $\phi_1$ and $\phi_2$ are ...
user avatar
  • 11
0 votes
1 answer
23 views

Linear Least Squares vs Ordinary Least Squares

What is the difference between ordinary LS and linear least squares? From wikipedia seems like linear least squares is a general category that involves using least squares to approximate linear ...
user avatar
  • 93
0 votes
1 answer
28 views

Linear regression has good performance in validation set despite not meeting the linearity assumption

I have a dataset with about 8000 samples and 18 predictors (16 continuous, 2 categorical). I am trying fit a linear regression, but despite trying multiple transformations, I can't make it meet the ...
user avatar
  • 33
0 votes
1 answer
28 views

Covariance of Least Squares Estimators

I am trying to solve the problem shown below: To solve (a), I defined the sum of squared errors $f(\hat{\beta}) = \sum_{k = 1}^{n} (y_k - \hat{\beta})^2$. This allows us to identify the least-squares ...
user avatar
0 votes
0 answers
29 views

Estimating the coefficients of a non-linear regression

I am trying to estimate the coefficients $\lambda, \alpha, \beta_1, \beta_2, \gamma, \eta$ in the below equation using Python and some financial data $$ \lambda \times \text{(participation %)} \times \...
user avatar
3 votes
1 answer
45 views

When is least squares better than reduced major axis?

Consider two linear regression methods: least squares regression (LSR) reduced major axis (RMA) I know the definitions of both regression methods but I would like to know when is the LSR better than ...
user avatar
1 vote
1 answer
16 views

Comparing Activities with regression

I am trying to find out what is the effect of activities(like jumping, weight lifting etc.) on behavior (such as attitude towards participating in a marathon). (sample size of 60 observations for each ...
user avatar
  • 11
0 votes
0 answers
21 views

What happens if the "coefficients" in the data generating process are correlated with the variance of the error term?

Suppose we are interested in estimating a regression of the form $$ y = \beta x + \epsilon $$ but in the data generating process, $\beta$ is decreasing in $\mathbb{E}[\epsilon^2]$. For example, there ...
user avatar
1 vote
1 answer
46 views

Why OLS perform better than LASSO?

I am comparing OLS and LASSO regression for survey data. I have n>p, but I think my data is high-dimensional data as the p is 3000 and n is 48000. I am using k cross-validation. The results are ...
user avatar
  • 11
0 votes
0 answers
13 views

Can introducing time fixed effects variable into a PanelOLS decrease overall and between R^2?

I am trying to find if there is a relationship between the number of people employed by the tech industry within a city and wages in that city. I ran two Linear Regressions on my data. The first one ...
user avatar
0 votes
0 answers
7 views

Distinguishing between effects of two variables on y

Assume that we have the linear regression model: $$ y=\beta+\beta_{1}x+\epsilon $$ We estimate the model by OLS, and we get $\hat{\beta}_{1},$ However, there is another variable $z,$ which is both ...
user avatar
0 votes
0 answers
26 views

For all datasets with a binary outcome, will linear regression always yield betas with a smaller standard error compared to logistic regression?

Any cases where the betas' standard errors from logistic regression will be smaller than linear regression, after converting from log odds space to probability space?
user avatar
0 votes
0 answers
9 views

Is my survey data structured correctly to run OLS regression in R?

I have data from two surveys that were launched at the same time and am trying to find whether there is a statistically significant result on the number of respondents as a consequence of various ...
user avatar
0 votes
0 answers
9 views

GPD and GEV Fitting: Maximum Likelihood vs. Least Squares

I am trying to build a model based on real world data which involves fitting generalized extreme value distributions and generalized Pareto distributions. Most literature immediately turns to the ...
user avatar
1 vote
1 answer
40 views

Baseline variable in regression

I am currently looking at this paper: https://www.nature.com/articles/s41591-021-01487-3 The equation for (1) includes a variable for a baseline value. I am confused as to why they do this as I ...
user avatar
0 votes
0 answers
23 views

How do I derive the variance of OLS estimators when I have dummy explanatory variables?

I know this isn't the smartest question, however I need to derive the variance of the OLS estimators in a Simple Linear Regression Model when the explanatory variable is a dummy one and all the ...
user avatar
  • 1
0 votes
0 answers
7 views

Using IV when regressor is not endogenous

Suppose I have a single regressor model and the regressor itself is uncorrelated with the error term. If I were to use IV estimation to estimate the coefficient, would the estimate be incorrect, and ...
user avatar
0 votes
1 answer
26 views

Interpretation problems of linear model with no predictors

Let $Y=X\beta$, where $X=(\begin{matrix}1 & 1 & 1\end{matrix})^T$, $Y=(\begin{matrix}6 & 5 & 4\end{matrix})^T$ and $\beta=(\begin{matrix}\beta_0 \end{matrix})$. Now $X\beta=(\begin{...
user avatar
  • 125
2 votes
1 answer
49 views

How to interpret coefficients vs relative importance of variables in linear regression?

I am running a OLS regression and to identify factors driving online sales of a product. This analysis (conducted only for inference) is run on different countries with the same variables included for ...
user avatar
0 votes
0 answers
27 views

LSDV/Fixed Effect Model Interpretation Confirmation using RStudio

I don't specialize in statistics, and I'm not the best with data! So would anyone be able to confirm if my interpretation is correct? I performed an LSDV/Fixed Effect Model (in R) using the regions ...
user avatar
0 votes
0 answers
11 views

Linear Regression Model output in R saying: "2 not defined because of singularities" [duplicate]

I am running a linear regression model in R where soya beans yield is predicted by soya varieties (spike,...
user avatar
1 vote
1 answer
17 views

Cointegration test; model with different number of explanatory variables

I have run an ADF test on the residuals of an ARDL and a DOLS model to test for cointegration. I have 3 explanatory variables and 1 response variable. When I run the ADF test on the residuals on both ...
user avatar
1 vote
0 answers
38 views

OLS coefficient estimation with Poisson errors

Assume that in this regression $$ Y=\beta_0+\beta_1 x+\epsilon, $$ where $\epsilon$ follows a Poisson distribution. Using OLS, estimate $\beta_0,\beta_1$ and $\text{cov}(\beta_1,\beta_0)$. I am ...
user avatar
1 vote
1 answer
59 views

Variance of OLS estimator with binary treatment

I know that in general, given a (stacked) regression of the form $ y = X \beta + \epsilon$, where $\mathbb{V}(\epsilon_i) = \sigma^2 \forall i$, then letting $\hat{\beta}$ denote the OLS estimate of $\...
user avatar
-1 votes
0 answers
27 views

Unbiasedness and consistency of OLS in an AR(1) model with AR(1) residuals [duplicate]

consider equation 1 : , Now let , where the error component is iid with mean 0 and constant variance, and Is the OLS estimator of the coefficients in equation 1 unbiased and consistent under this ...
user avatar
1 vote
1 answer
37 views

Different number of observations after including control variables

I have two regression models. I am using paneled data on individuals from 2010 up to 2019. For some individuals, I have several years of observations, whereas for others, there are only 2 or so. The ...
user avatar
  • 41
1 vote
1 answer
48 views

Do I need to transform/standardise my dependent variable?

Attached are the results and the residual plot for my regression of control variables on CEO compensation (TDC1). When I look at the plot my main concerns are the outliers (which I checked to be ...
user avatar
1 vote
1 answer
26 views

Gradient of the second order term of Newton's Method

I know that Netwon's method can be pushed to the second order using the 1st Taylor expansion. However, how can I generalize Netwon's method to take x_0 as a vector and have the ability take the ...
user avatar
1 vote
0 answers
10 views

Parametric bootstrap *prediction* interval with heteroskedasticity and sandwich parameter covariance matrix

The sandwich estimator for OLS regressions where heteroskedasticity is suspected is $$ var(\hat\beta) = (X'X)^{-1}X'ee'X(X'X)^{-1} $$ If I want confidence intervals on predictions, I can just take ...
user avatar
  • 12.1k
1 vote
0 answers
11 views

How to derive least squares estimators from normal equations with work shown

My last question was closed, so I made a new one that is more relevant and has my full calculations thus far. I have a scenario where I have been given some information, such as normal equations from ...
user avatar
  • 65
1 vote
2 answers
88 views

How to obtain least squares when $X^TX$ cannot be inverted

This work is all theoretical and for school, so we were only provided this information to work with, no actual y values. I have a simple linear model I have been asked to translate into a matrix, ...
user avatar
  • 65
0 votes
0 answers
14 views

Estimating parameters GARCH model

Suppose I have the following model \begin{equation} R_{i t}-r_{t}=\alpha_{i}+\beta_{i}\left(R_{m t}-r_{t}\right)+s_{i} S M B_{t}+h_{i} H M L_{t}+\varepsilon_{i t} \label{eqn:egarch} \end{...
user avatar
  • 11

1
2 3 4 5
51