Questions tagged [least-squares]
Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.
2,697
questions
0
votes
0
answers
19
views
Least squares estimators with a restriction [closed]
Assume the model $ y_i = \beta_1 log(x_{1i}) + \beta_2 /x_{2i} + \epsilon_i$
with the restriction that $\beta_2 = 2 \beta_1$. Find the least-squares estimators of the regression coefficients.
I am ...
- 1
0
votes
0
answers
5
views
Why am I getting incorrect predictions from ols model applied on standard scaled data set? [closed]
I have applied standardization to my dataframe with the following code:
...
- 101
0
votes
0
answers
16
views
Ordered Logistic Regression or OLS
I have five employee-level variables.
Z1 = employee wage
Z2 = employee tenure
Z3 = employee age
Y = employee perception of fairness (ordinal survey item with 5 = very fair... 1 = very unfair)
X = ...
- 1
1
vote
1
answer
37
views
Why would bootstrap OLS standard errors differ from ML estimate?
Let's say I have a regression dataset (paired x and y) such that the response variable (y) has an unknown distribution (but definitely not Gaussian) and is large enough such that the central limit ...
- 65
1
vote
0
answers
18
views
OLS assumptions for weighted errors
As far as I know, under satisfied assumptions for OLS, the estimates acquire qualities like BLUE, MVUE, MLE.
But in the case where there is a priori knowledge of the influence of each data point, it ...
0
votes
0
answers
23
views
Proof of a statement about the variance of an OLS estimator
I am currently taking an econometrics course and in the lecture notes there is a statement about the variance of an OLS estimator that I am unable to prove. The statement is as follows:
Suppose the ...
- 1
1
vote
1
answer
29
views
Omitted variable problem
I'm studying the cases in which the endogeneity problem arises in OLS regression.
Suppose we have the following population equation:
$y=\beta_0 +\beta_1 x_1 + ... + \beta_k x_k + \gamma q + \epsilon$
...
- 99
0
votes
0
answers
14
views
When the endogenous variable in a regression is population changes by age group, do I run a separate regression for each age group?
I am running a regression on how trade shocks affect population by age group at the county level. We bin ages into three age groups. I am confused about whether I should run one big regression, like ...
1
vote
0
answers
20
views
If we have a binary variable in our linear regression, the VIF for its coefficient estimate uses the $R^2$ of a linear probability model. What gives?
The variance inflation factor (VIF) in an ordinary least squares linear regression coefficient is calculated using the $R^2$ of a linear model that uses the other features to predict the feature to ...
- 47.2k
0
votes
1
answer
14
views
Different time lags for different explanatory variables
I have built a Dynamic Model with an AR(1) term and some other explanatory variables for example inflation, GDP and interest rate. I have used different time lags for the different explanatory ...
0
votes
0
answers
10
views
Testing implications of Conditional Mean Independence
In some empirical studies as a validation exercise some people regress some variables on the variable of interest controlling for key control variables. The reason for doing this I think comes from ...
- 23
5
votes
3
answers
98
views
What does it mean for $\hat\beta_1$ and $\hat\beta_0$ to have a variance?
With regarding to OLS estimators why $\hat\beta_1$ and $\hat\beta_0$ have a variance?
- 51
2
votes
2
answers
54
views
Show that $E [u] = 0$ and $cov(u,x_j)=0$ does not imply $E[u|x]=0$
In the regression model
$$y= x'\beta + u, \quad x = (1, x_2,...,x_K)$$
with
$$E[u |x]=0,$$
we know that it implies: $E [u] = 0$ and $cov(u,x_j)=0$, for $j=1,...,K$.
I think that the reciprocal is not ...
2
votes
0
answers
85
views
Why is i.i.d. an OLS assumption?
Assume the following linear relationship:
$Y_i = \beta_0 + \beta_1 X_i + u_i$, where $Y_i$ is the dependent variable, $X_i$ a single independent variable and $u_i$ the error term.
According to Stock &...
- 345
1
vote
0
answers
22
views
R squared for a regression plane without observations
Assuming we have three random variables $X$, $Y$, and $Z$, and we want to estimate a least squares regression plane of the form $Z = a + bX + cY$. We do not know the individual observations, but we ...
1
vote
3
answers
124
views
Performing 3 multivariate linear regressions at once
I have 3 variables X, Y and Z. I want to perform 3 OLS regressions: X dependent on Y and Z, Y dependent on X and Z, and Z dependent on X and Y.
Instead of doing the 3 of them sepparately, I want to ...
- 83
1
vote
0
answers
33
views
OLS regression with fixed effects when dependent variables are measured differently by group
I have data where each observation is a region within a country, and each observation falls into one of two large groups (each group makes up half the dataset). Each group corresponds to a seven-year ...
- 11
3
votes
1
answer
54
views
Cannot understand why total SS＝explained SS+unexplained SS
I cannot understand why total SS＝explained SS+unexplained SS because geometrically the sum of two small squares is not equal to a big square.
I wish someone could explain that to me. Thank you.
- 31
1
vote
0
answers
27
views
Estimating differences slopes between groups with y as the computed slope
I'm estimating a basic OLS model, y ~ x, and I want to know the difference in the slope between two groups. Here is a simulated example in R:
...
- 111
0
votes
0
answers
16
views
Simultaneity biases
I am trying to figure out whether I can trust OLS results in this situation.
I have possibly simultaneous equations that follow the models:
$ y = k_1 + \gamma_1 x + \beta_1 z + u_1 $
$ z = k_2 + \...
- 11
1
vote
2
answers
65
views
What is this equation for?
$$t = \frac{r}{\sqrt{\frac{1-r^2}{n-2}}} \sim t_{n - 2}$$
with $$r = \frac{S_{xy}}{\sqrt{SC_{xx} SC_{yy}}}$$
I've just found this equation with no direct data to of what can I get from it, I couldn't ...
- 240
1
vote
1
answer
38
views
Not taking the log of a variable with lots of 0 observations
I have some variables in an OLS that are very skewed to the right, so it could make sense to the log transformation of it, however there are lots of 0s in the variable, so that would mean I have to ...
- 133
7
votes
2
answers
298
views
Standard Error of Noise Variance in Least Squares
Suppose I'm doing ordinary least squares with homoskedastic errors, so something like:
$$
y = X \beta + \epsilon
$$
where $\epsilon \sim \mathcal{N}(0,\sigma^2)$.
I know how to estimate the expected ...
- 225
4
votes
3
answers
96
views
What would be the effect of modeling a binary predictor in an OLS model as [-1, 1] instead of [0, 1]?
I am using an OLS model to predict a continuous variable using several continuous predictors and one binary categorical predictor. I know that usually binary variables are modeled as [0, 1], but I am ...
- 141
0
votes
1
answer
38
views
Least squares estimator of normally distributed observations
I am trying to comprehend what the authors are asking me to show in this following exercise problem from Hogg, McKean and Craig's book titled "Introduction to Mathematical Statistics."
3.5....
0
votes
0
answers
10
views
Looking for specifications for OLS when time series are not stationary
Okay, so I have a quick question about the OLS specification when timeseries is not stationary.
I have two time series variable with 64 observations over time. So there are 64 rows and 3 columns in my ...
2
votes
1
answer
50
views
Frisch–Waugh–Lovell (FWL) Involving Interaction Terms in OLS
Suppose that I have two structural equations:
$$ X = \alpha_1W + \epsilon_1$$
$$ Y = \beta_1X + \beta_2W + \beta_3XW + \epsilon_2$$
In other words, $W \sim N(0,1)$ is a confounder and effect modifier. ...
- 23
0
votes
1
answer
28
views
OLS model specification that includes all dummy variables with a predetermined coefficient
I'm working with a OLS model that includes dummy variables (quarters of year). Here's what I would specify it:
$$y = \beta X + \gamma_1Q_1 + \gamma_2Q_2 + \gamma_3Q_3 + \epsilon$$
However, in the ...
- 679
4
votes
2
answers
80
views
Which is the best backtransformation correction method for log(outcome) predictions?
I am using OLS on log(outcome) to predict future outcomes. My question is which method of correction to backtransformed predictions will least bias the modeled effect of change in predictor values.
...
- 133
1
vote
0
answers
37
views
OLS - The relationship between "minimizing SSR" and "the ration between cov(X,Y) and Var(X)" [closed]
Question
What would be the intuitive explanation for the slope of Ordinary Least Squares(OLS), which is $\frac{cov(X,Y)}{var(X)}$ contributes minimizing the sum of squared residuals?
In the same ...
- 327
1
vote
1
answer
70
views
Least Squares Estimation for CRD experiments
Setup
Let us use the following model for a Completely Randomised Designed (CRD) experiment:
$$ Y_{ij}= \mu + \tau_i + \epsilon_{ij}$$
where the errors $\epsilon _{ij}$ ~ Normal ($0, \sigma ^2 $) and ...
- 173
1
vote
1
answer
60
views
OLS model and random effects model generate identical outcomes
Using R (package plm), I ran OLS and random effects model to compare the effect of marriage on earnings based on the youth survey longitudinal data.
Strangely, both ...
- 305
0
votes
0
answers
43
views
Consistency of OLS estimator
Under some assumptions, OLS estimator is consistent to true value. With all the assumptions hold, now I would like to know if OLS estimator calculated from subset of sample is still consistent or not.
...
- 1
3
votes
3
answers
215
views
Demeaning variables in OLS
I am analysing some data in R where I have information on $y$ and $x$.
When I run $y = \alpha + \beta\cdot x$ I get the same coefficient on $x$ (i.e. $\beta$) as ...
- 55
0
votes
0
answers
14
views
What does reduced major axis (RMA) look like when the X and Y variables aren't correlated?
I'm currently learning about the topics of correlation and regression and my book states in the section about using reduced major axes (RMA):
If there is no relationship between the two variables (X ...
- 101
2
votes
1
answer
50
views
What is MS Residual, and how it differs from the common aleatory error?
Hello I have two questions.
I am trying to understand this formulas. But I cant understand how the MS Residual is different from the common aleatory error
Another thing, I just labeled each value ...
- 240
1
vote
1
answer
95
views
Gauss Markov Theorem for linear combinations [duplicate]
I know that the Gauss Markov theorem implies that under some conditions, OLS estimates have the smallest variance of all unbiased linear estimators.
In particular if I have a model like $ y = \alpha + ...
- 55
1
vote
0
answers
41
views
How to adjust team win probability based on in in game win probability
I have a formula to calculate each team's win probability based on their win rates and home court advantage. So lets say I use it to determine that team A has 45% chance to win and team B has 55% ...
- 11
0
votes
1
answer
38
views
Least squares estimator for parameter p in binomial distribution
I am trying to find the least square estimator for the parameter $p$ in $Bin(n,p)$ but is it even possible? Isn't it the same as finding it using MLE?
- 1
1
vote
1
answer
35
views
What is the difference between standard errors and residuals in OLS?
I'm trying to get a deeper understanding of how OLS works. One thing that I thought I understood is the difference between standard errors and residuals.
Here are two definitions
Standard errors: The ...
- 165
0
votes
1
answer
71
views
Price elasticity of demand estimation with single store level data
I have weekly data per single SKU and for more than 500 point of sales. Data embeds base price, quanity sold, holidays, temperature, market activities (cut-price, display, leaflets) and so on.
I want ...
- 51
8
votes
1
answer
332
views
Least squares problem penalising the number of non-zero coefficients
Consider a constrained least squares problem of minimizing $(y-X\beta)^T(y-X\beta)$ subject to a constraint or penalty on the number of non-zero coefficients $\beta_i$. This seems related to LASSO or ...
- 320
1
vote
0
answers
52
views
How to structure combinations of dataframes for regression, without corruption/loss?
I have a data set, redacted sample below. My goal is linear regression. My question is: Have I created unintended results, due to how I structured the df, using concat and/or div?
For example, ...
- 11
0
votes
0
answers
20
views
Have I implemented WLS correctly?
I have to predict the amount of charitable donations with my dataset. The OLS model I initially used is this:
...
- 1
0
votes
0
answers
13
views
for linear regression, when is MLE the same as least squares? [duplicate]
Under what conditions is MLE the same as the least squares estimate for ordinary linear regression?
I have seen statements saying that these two methods are not entirely the same. But so far, using ...
- 8,457
0
votes
0
answers
27
views
Interpretation of rescaled coefficients in OLS
Context
To make the coefficients of regression coefficients comparable, one usually rescales by the standard deviation (see case 3 below). This means that the different regression coefficients can be ...
- 101
5
votes
3
answers
584
views
OLS - Why coefficient Beta has Normal Distribution but not t-Distribution
I have a little trouble understanding the solution to the following problem. To my understanding, coefficients in OLS should follow t-distribution. However, the solution says it follows Normal.
Please ...
- 163
1
vote
0
answers
29
views
How many control variables can I use in experiment / observational study without overfitting?
Suppose I run a regression looking at the correlation of a training program on wages and I include dummies for different levels of for age, education and industry:
$$
wage_i = \beta_0 + \beta_1 ...
- 33
0
votes
0
answers
11
views
Find an estimator of linear regression when errors variance is correlated with one of the K regressors
I need to answer to the following problem:
In an heteroschedasticity setting, let $n$ be the index of the n-th statistical unit with $n=1, \dots, N$.
Suppose a multiple linear regression setting with ...
1
vote
1
answer
36
views
Unbiasedness of OLS when using averages instead of full data?
So my professor used the following so show that using averages instead of the full data set in OLS Regression will not cause any bias in our estimator. I understand everything up to the penultimate ...
