Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least squares and least squares is the idea underlying the use of mean-squared-error (MSE) as a way of evaluating an estimator.

1
vote
2answers
37 views

Is it valid to iterate over every permutation of a regression specification and compute an “average significance?”

I had an idea, and was wondering if it's ever done and, if so, how to do it in an appropriate manner. Let's say I run an OLS model and the results come back significant. There is discussion as to ...
2
votes
0answers
19 views

Show positive semi-definiteness of co-variance differences

I am having difficulty checking if $$ (A'D^{-1}A)(A'D^{-1}BD^{-1}A)^{-1}(A'D^{-1}A) - (A'A)(A'BA)^{-1}(A'A) \succeq 0 $$ where $B\succ 0$ is a square co-variance matrix and $D$ is a diagonal matrix ...
0
votes
0answers
11 views

How to obtain a trend estimate using only ols?

I have data on $y$ and $t$ and I want to estimate the trend in $y$ that is supposedly given by relation $y=a(1-b e^{t/τ})$. I want to use linear regression on some transformed version of the model in ...
0
votes
1answer
21 views

Derivation of Formula for linear regression with data points forming a circle

Suppose we have one dimensional data with following conditions.1. The value of this single feature lies in the closed interval [-a,a].2. The associated target value with each data also lies in the ...
0
votes
0answers
14 views

How would I figure out - TSS of Y, Yi and XiYi from the following information?

I came across an analogous question when revising and have no clue how to approach it. The Given information is ΣXi= 20, ΣYi=40 Σ(Xi-x̅)²= 40, Σ(Xi-x̅)(Yi-nȳ)=20 and n=20. The question requires ...
0
votes
0answers
10 views

Comparing the output and fit of an OLS and a Tobit model, and comparing Tobit models

I am trying to estimate a quite complicated model (many variables with different structures), with a limited dependent variable, which ranges from 0-100% with about 45% of the sample having an ...
0
votes
0answers
21 views

Cannot replicate a 95% CI using: Beta ±1.96*SE [duplicate]

I am having a lot of difficulty figuring out why I cannot calculate a 95% CI by hand with one set of data; but I can using the same process with a different set of data. Drawing on OLS results; I am ...
0
votes
0answers
9 views

Least squares and identificability condition

Let a discrete-time system (which is minimal) with input $u \in \mathbb{R}^m$ and state $x \in \mathbb{R}^n$ be $ x_{k+1} = [x^T_k \quad u^T_k]\begin{bmatrix} A^T \\ B^T \end{bmatrix} + v^T_k $ ...
0
votes
1answer
16 views

Cluster-robust standard errors in panel data analysis

In a simple panel data analysis with data on 64 firms over 8 years, I use cluster-robust standard errors (at the firm level) to evaluate significance of coefficients. I observe important differences ...
0
votes
0answers
12 views

upward/downward bias of negative variable

If I have a variable that, considering some omitted factor, should have fallen by a higher amount than when it is not there - would that be a downward bias? I.e. the decrease is not large enough, so ...
1
vote
0answers
27 views

Degrees of freedom in OLS regresison vs Bootstrap

I understand that in OLS, the degrees of freedom for estimating the variance of the residuals is n-q-1. We loose q+1 degrees because they are "used" to analytically determine the q parameters and 1 ...
0
votes
0answers
11 views

How to motivate a POLS?

How would you justify the usage of Pooled OLS regression instead of Fixed effects? If I am calculating just correlation between two phenomena, may I get rid of these fixed effects? May I choose to ...
2
votes
1answer
43 views

Does $\hat{\beta}=(X′X)^{−1}X′ \ Y$ simplify further?

As far as I know $(AB)^{-1}=B^{-1}A^{-1}$ and matrix multiplication is associative so: $$(X′ X)^{−1}X′=(X^{-1})(X')^{-1}(X')=X^{-1}.$$ What am I doing wrong?
1
vote
1answer
21 views

How to obtain the inverse of the variance covariance matrix of GLS (Random Effects Model)

In the standard GLS set up how do you find the inverse of the variance covariance matrix? $$y _ { i t } = \beta _ { 0 } + x _ { i t } ^ { \prime } \beta + \alpha _ { i } + u _ { i t } \hspace{35pt} u ...
0
votes
1answer
18 views

Log-normalization of predictors

I have the following dependent and independent variables for my linear regression model. Since they are all in different scales (some of the are % others continuous variables), I was suggested to take ...
0
votes
1answer
19 views

Does a panel regression model make sense for my data?

This might be a bit of a newbish question, but I recently picked up a forecasting project at my job, and I'm trying to figure out whether it makes sense to run a panel regression like a Fixed Effects ...
0
votes
0answers
26 views

Why does the variance of the OLS estimator goes to zero as n increases while it is a random variable?

I have a question regarding the OLS estimator. In my class, the teacher asked us to critic this comment: “We write the unconditional variance of the OLS as $V(\hat{b}^{ols}) = 1/N 𝜎^2 E(X_i'X_i )^{−...
0
votes
0answers
13 views

Dynamic Pooled OLS: does it make sense?

I have data on 25 countries for about 35 years. I want to estimate a POLS as I am not looking for a causal interpretation but just sttistical correlation between two phenomena. That's why I will not ...
0
votes
0answers
7 views

On dichotomizing a continuous variable and how it affects MSE of OLS

Recently I have learned about the practice of dichotomizing a continuous independent variable (or maybe even discretize it into more than 2 categories), and then run a predictive model (e.g. multiple ...
0
votes
1answer
11 views

Significance of the sum of the main effect and interaction term

Consider a simple linear regression with an interaction term: $Y=b_0 + b_1X +b_2Z+b_3XZ$ where $X$ is continuous and $Z$ is a dummy. I want to find out whether $X$ has a significant impact on $Y$ ...
0
votes
0answers
28 views

How do we compute OLS coefficients from Sum of Squares in multiple linear regression?

Suppose we have a variable y which is dependent on 2 variables say x1 and x2. Then I can understand how we can compute Sum of squares due to x1 and x2. For instance we may have Type I Sum of squares ...
0
votes
0answers
8 views

LS classifier that is based on the sum of error squares criterion

Is it possible given a set of points from two classes to determine the LS classifier that is based on the sum of error squares criterion? I don't ask if it is efficient if it is possible and if yes ...
0
votes
0answers
43 views

How to prove variance of OLS estimator in matrix form?

I am reading Wooldridge's Introductory Econometrics (2000), don't judge me, old version = cheap second hand book, and in the page P94 Theorem 3.2 of Multiple Regression Analysis, it says that: $$ Var(...
2
votes
0answers
58 views

Negative values for OLS variance

I am currently writing some code which performs regression and have noticed that when I calculate variance of $c\hat{\beta}$ I am sometimes on some datasets getting negative values. The variance is ...
0
votes
1answer
41 views

Implications of strict exogeneity for OLS in time series

Zero Conditional Mean (ZCM), or Strict Exogeneity, is given by: $E[u|X]=0$ Equivalently, $E[u_t|X]=0, t=1,...,T$ Is it true that this implies: Zero Unconditional Mean: $E[u_t]=0, \forall t$ ...
2
votes
0answers
29 views

Comparison between coefficients of two fixed effects models with same sample

I have a problem. I have two models of OLS regressions with time and group fixed effects of a crosscountry paneldata analysis: $$Y_i = b_1 X_i + b_2 D_1 Z_i + b_3 D_2 Z_i\tag 1$$ and $$Y_i = b_1 ...
0
votes
0answers
31 views

Does standard error affected by the coefficient?

I make a comparison on ridge regression and OLS using simulation. As i set my correlation as 0.9, which is high, i expect the standard error of ridge regression to be low. However, it is not. ...
2
votes
0answers
28 views

Intuition behind White's Estimators/ Heteroscedasticity-consistent Standard Errors

For a medical study I am trying to understand the intuition behind heteroscedasticity-consistent standard errors. I know that it can be used, when in OLS regression residuals are heteroscedastic. By ...
3
votes
1answer
32 views

Obtain global linear regression estimate from subsamples

I want to estimate $\widehat\beta$ in a simple linear regression with scikit. $$y = X \beta + \varepsilon$$ The problem is that the dimension of the complete $X$ is too large to fit into memory. Is ...
0
votes
0answers
17 views

Comparing the mean of predicted values for a misspecified model against the mean of the observed values

I have a regression that I have run on average ratings for some products (dependent variable) and their characteristics (Model 1). I have reason to believe there is a prejudice against a specific set ...
1
vote
0answers
32 views

How to verify the “random sampling” Gauss-Markov Assumption with Stata (or anything else)?

According to the book I am using, Introductory Econometrics by J.M. Wooldridge, there are 5 Gauss-Markov assumptions necessary to obtain BLUE. However, by looking in other literature, there is one ...
1
vote
1answer
55 views

linear regression predicts lower than expected

I am trying to predict first term GPA for college students based on a number of incoming factors (high school gpa, placement test, year). This isn't the overall model just a simpler one. The first ...
1
vote
2answers
31 views

Linearization of multiplicative and exponential regression models and their OLS estimation

So far I have been under the impression that you can "linearize" multiplicative models of the form (1) $y=\alpha * \beta_1x_1 * \beta_2x_2 * \beta_3x_3 $ and exponential models of the form (2) $y=\...
1
vote
0answers
12 views

Simulating size distrotion with a VAR(1) model

Firstly, I am not asking for code, I would like the intution of how I would do this. I am testing for size distortion. I have estimated and VAR(1) model and I have the parameters. I want to ...
1
vote
0answers
23 views

Consistency of an OLS estimator in time series

In the ARMA(2,1) time series $y_t = \beta_0+\beta_1y_{t-1}+\beta_2y_{t-2}+ u_t + \phi_1u_{t-2}$ $u_t$ and $u_{t-2}$ are white noise shocks. The time series are stationary and ergodic. In the ...
2
votes
0answers
22 views

Residualizing in OLS

Consider a linear regression $$ y_i = A_i' \theta + B_i' \psi + \epsilon_i $$ There's a "trick" to find the parameters $\theta$ and $\psi$ in a stepwise fashion. First minimizing squared error wrt....
0
votes
1answer
29 views

Without testing: when are error terms possibly homoscedastic?

I am facing the following study: In the 1980's, Tennessee conducted an experiment in which kindergarten students were randomly assigned to regular and ...
-1
votes
0answers
15 views

Beta hat marginal distribution in Strong OLS setting

If in OLS setting I assume: Epsilon|X is normally distributed (mean= 0; variance= sigma^2 * Identity matrix) We have that the estimated coefficients (each "beta hat j") is conditional on X normally ...
0
votes
1answer
24 views

How to fix and understand linearity

The model I have run is a simple multiple linear regression. The model looks like a great fit, but R is telling me otherwise. My question is 3 fold. 1) How do we estimate linearity (not visually) 2) ...
2
votes
0answers
38 views

Can I average out a constant (intercept) in OLS regression?

I have a OLS regression in the form: $$Y_t=\alpha +\beta X_{t-1}+\varepsilon_{t}$$ Can I average out the constant during the OLS estimation/derivation and report, $$y_t=\beta X_{t-1}+\varepsilon_{...
3
votes
0answers
36 views

Dealing with autocorrelation using Generalized Least Squares

I have a time series data set where the auto correlation of the residuals follow an exponential decay. I was wondering how I should deal with this? I would like to fit a linear model and have tried ...
0
votes
1answer
24 views

Is it possible to add the standard errors of 2 groups together to obtain the standard error of the 2 groups combined

I am trying to recreate the results in this table. The results have been obtained by difference in difference estimation. I can obtain values from all columns except for column 5 and 6. Column 5 says ...
1
vote
1answer
52 views

OLS Population Orthogonality Condition Proof

In the OLS model, we assume that $E(X'U)=0$ (with $u$ being the error term), which comes from $E(U|X=x)=0$, providing us that $E(U)=0$ and $cov(x_i, u)=0$ $\forall x_i$. I understand this argument ...
0
votes
0answers
26 views

Ordinary least squares does not optimize error

I'm trying to use polynomial regression to fit the curve $$X = [0, 1]$$ $$Y = \sin(2 \pi X) + \epsilon$$ where $\epsilon$ is normally distributed with the same $\sigma$ for all $X$ For every value ...
2
votes
0answers
27 views

Baseline adjustment in a change model given bidirectional causation

I come from a non-statistical background and am trying to wrap my head around whether baseline adjustment is necessary in a change model when analysed using OLS regression. I am considering different ...
1
vote
1answer
35 views

Uncorrelated errors with the regressor in a reduced form VAR

I have a reduced form VAR $$\begin{equation} y_t = c_o + A y_{t-1} + \epsilon_t \end{equation}$$ Where, $y_t \in \mathbb{R}^2$, $A$ is a $2$X$2$ matrix and $$\begin{equation} E(\epsilon_t \...
3
votes
1answer
88 views

Controlling for a variable in OLS - Stratification and Reaggregation. Simple Example

In his engrossing book "Naked Statistics" Charles Wheelan begins to explain how controlling for variables works by stratifying the sample. However, he stops short of explaining the reaggregation, ...
0
votes
0answers
25 views

Solving correlation between explanatory variables using instrumental variables

I am currently stuck on a task where I am interested in estimating the production function for agricultural output as follows: \begin{equation} y_{i} = x_{i}\beta + \alpha_i + \epsilon_{i} \end{...
0
votes
0answers
15 views

Transforming panel data OLS into cross-sectional data model

I am currently stuck on a task where I am interested in estimating the production function for agricultural output using panel data as follows: \begin{equation} y_{it} = x_{it}\beta + \alpha_i + \...
0
votes
0answers
23 views

R square for fixed model is worse than OLS in panel model

I am analysing panel data and using the plm package in R. I'm using the plmtest, pFtest, phtest functions which indicate to me that fixed effects model should be used over the pooled OLS and random ...