Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Refers to a general estimation technique that selects the parameter value to minimize the squared difference between two quantities, such as the observed value of a variable, and the expected value of that observation conditioned on the parameter value. Gaussian linear models are fit by least ...

1
vote
0answers
7 views

VIF Drops Significantly When I Delete Some Dummy Variables

I have a a category variable (Fruit) that I converted to dummy variables: columns Apple, ...
1
vote
2answers
35 views

Variance of $\hat{\beta}$ when assuming heteroskedastic error terms

I am wondering why, when we are assuming heteroskedastic error terms: $Var(\hat{\beta}|X) = (X'X)^{-1}X'E[\varepsilon \varepsilon'|X] X(X'X)^{-1}$ simplifies to $Var(\hat{\beta}|X) = (\sum{x_ix_i'...
0
votes
0answers
9 views

Robust Regression in MATLAB's robustfit: what is the optimal weight function to tackle heteroskedasticity?

I'm currently performing a linear regression analysis and encountered a fair amount of heteroskedasticity. Increases in predicted values go along with decreases in residual variance. Otherwise, the ...
0
votes
0answers
52 views

Stuck on a term in $\operatorname{Var}\left[ \widehat{\beta}_0 \right]$ proof

So I was trying to prove that $\operatorname{Var}[\hat{\beta}_0] = \dfrac{\sigma^2n^{-1} \sum{(x_i)^2}}{\sum{(x_i-\bar{x}})^2}$ And I got stuck with the part $\dfrac{-2\bar{x}}{\sum{(x_i-\bar{x})^2}} ...
0
votes
0answers
26 views

How to calculate the variance of y in in the OLS model? [on hold]

When calculating the variance of the coefficients we get something like this: β̂ =(X′X)^−1X′y.(X is the design matrix and β̂ is the coefficient vector) and thus Var(β̂)=(X′X)^−1X′σ^2IX(X′X)^−1=σ^2(...
0
votes
0answers
9 views

Determining the Relationship Between Monte Carlo Breaks and Model Volatility

I'm looking for a statistical test to understand the relationship (if any) between the model volatilities of a stochastic process, and the occurrence of a'break', defined as an instance when an ...
0
votes
1answer
49 views

Expected value of squared least squares estimator

I am trying to prove $E(\hat{\beta} '\hat{\beta}) = \beta'\beta+\sigma^2 *\sum_{k=1}^K\lambda_k^{-1}$ where $\lambda_k$ denotes the eigenvalues of the matrix $(X'X)$ with dimensions $K\times K$. $\...
0
votes
0answers
24 views

Regression equation passing through the origin

I'm going through Multiple Choice Questions of Basic Econometrics by Gujarati. There is a question which states that: It is a simple two-variable regression: Any regression equation written in its ...
1
vote
0answers
24 views

OLS R-Squared Drops by 15% When Constant / Intercept is Removed?

I ran an Ordinary Least Squares model and found the constant / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the ...
0
votes
0answers
14 views

Find the ML and LSE estimation of b in a Rayleigh distribution

Im having a hard time with a homework for a Statistical Course im taking. The question is as follows. " A Rayleigh distributed stochastic variable X have the density function $$ f_X(x)=\frac{x}{b^2} e^...
1
vote
1answer
35 views

How to find the OLS estimator of variance of error

Given the Linear Regression model $y=X\beta+\epsilon$, where $\epsilon \sim D(0_n,\sigma^2 I_n)$ and $D$ is some distribution. How to find the OLS estimator of $\sigma^2$. I know that the sum of ...
0
votes
0answers
20 views

Time dummies in Linear Panel Data Models and Unobserved Effects Models

Wooldridge (2002, p. 129) says that with (independently) pooled cross sections over time (where different random samples are collected at different points in time - no individuals are observed more ...
1
vote
0answers
48 views

Proof Verification: $\tilde{\beta_1}$ is an unbiased estimator of $\beta_1$ obtained by assuming intercept is zero

Consider the standard simple regression model $y= \beta_o + \beta_1 x +u$ under the Gauss-Markov Assumptions SLR.1 through SLR.5. Let $\tilde{\beta_1}$ be the estimator for $\beta_1$ obtained by ...
1
vote
0answers
17 views

Using fitted lagged variable as dependent variable in a new regression?

Suppose I have regression like this; $y[t] = a * exp(b*y[t-1])$ From this regression, I get; $\bar y = y - residuals$ What happens if I regress a new regression like this? $y[t] = c * exp(d*ybar[t-...
2
votes
2answers
31 views

Intuitive explanation from regression coefficient estimate formula

Can someone provide an intuitive explanation of why the OLS regression estimate, of y=a+bx, b have the form b=cov(x,y)/V(x). How intuitively are the covariance and variance related in this?
1
vote
1answer
26 views

Assumptions of linear fit; linearity and homoscedasticity

I'm reading about the assumptions of taking a linear fit between two variables from here, and that source says: For diagnosing non-linearity: nonlinearity is usually most evident in a plot of ...
1
vote
1answer
34 views

Unbiasedness and Consistency of DID estimator - pooled cross sections over time

Consider two time periods, where In time-period 1: a random sample is collected for group 1 (control) and a random sample is collected for group 2 (treated). In time-period 2: a random sample is ...
0
votes
1answer
38 views

What is the best way to test and validate a multivariate regression using OLS?

I am implementing a multivariate regression from scratch using Ordinary Least Squares to get the weights. I noticed that this method does not have any hyperparameters to tweak, so I am not sure what I ...
1
vote
0answers
89 views

Expected value of the residuals

How would one prove that the expected value of the residuals from OLS regression is zero? I will make two cases. In the first case I treat $X_i$ as random and in the second case I treat it is non-...
0
votes
0answers
21 views

ANOVA tables are exactly the same using OLS and Poisson regression?

I have fit two models in R, as follows: m1 = lm(y ~ x * z) m2 = glm(y ~ x * z, family="poisson") Where y is the dependent ...
0
votes
1answer
13 views

Degrees of freedom correction in estimation of AR(p) process

Assume that I have a process $y_t$ such that $$y_t = c + \phi_1 y_{t-1} + \ldots + \phi_p y_{t-p} + u_t$$ where $u_t$ is i.i.d. white noise such that $E[u_t] = 0, \forall t$ and $E[u_t u_s]$ is equal ...
1
vote
0answers
32 views

Is fixed effects (within estimator) estimated using OLS?

In a paper I submitted to a journal, I estimated a fixed effects panel model with regional fixed effects. The reviewer of my paper in his comments said my estimation was bad because I "simply used OLS ...
2
votes
1answer
31 views

Does regularization in regression help with numerics when the data matrix is not full rank?

I am trying to get some intuition around regression when the data matrix $A$ is not full rank in the following regression/least squares problem: $$y=Ax+b$$ where $y \in \mathbb{R}^n$, $A \in \mathbb{...
1
vote
1answer
27 views

Decomposition of vector into product of a function on a matrix and a function on a vector - Possible? [closed]

Say I have access to $N$-dim vector $Y$, $N \times p$ matrix $X$, and $q$-dim vector $Z$. Ultimately, I would like to learn the functions $g,f$ in: $\underset{N\times1}{\underbrace{Y}}=\underset{N\...
1
vote
0answers
20 views

Long T Small N causal time series ols regression analysis ? can i use panel data? [closed]

I have N=28 AND T=260 WEEKLY DATA and i want to check impact of two variables on stock volatility.i proposed that one variable moderates others effect on stock volatility . I was using time series ...
3
votes
3answers
82 views

Finding OLS estimator for $\beta$ where $y_i=\beta+ 2 \beta x_i+\epsilon_i$

Consider the following model with the usual OLS assumptions: $\epsilon_i$ are uncorrelated random variables with mean zero and constant variance $\sigma^2$. $$y_i=\beta+ 2 \beta x_i+\epsilon_i$$ $(...
1
vote
0answers
13 views

Normality assumption for t/z tests

I have what is probably a silly question regarding normality assumptions for t/Z tests. As I understand, t/z tests require that sample data was obtained from populations following a normal ...
0
votes
0answers
10 views

How to estimate covariance matrix given a sample of auto-correlated panel data?

For example, I have K factors in T time periods, thus my data X is KxT size. If not considering the auto-correlation in each factor time series, the covariance matrix can be simply estimated by $Cov(X)...
2
votes
1answer
30 views

Show that solution to cubic smoothing spline reduces to regular least squares minimization as $\lambda$ approaches infinity

I am asked to show that the solution to a smoothing splines problem of the form $$ \text{PRSS}(f,\lambda) = \sum_{i=1}^N\left[y_i-f(x_i)\right]^2 + \lambda \int f''(t)^2 dt, $$ with $$ f(x) = \sum_{j=...
0
votes
0answers
11 views

R/S-Analysis: Hurst Coefficient Using Least Squares Method

The application I am working on (it is an image processing program) needs to calculate, given an m-by-m matrix of integers, the so-called Hurst coefficient for that matrix, considering it as a time-...
1
vote
1answer
31 views

Role of random sample assumption in consistency of OLS estimator

I guess in part what this all amounts to is what does the assumption {(x_i,y_i) : i=1,2,...,n} being i.i.d. imply about the i.i.d-ness of functions of it? I am confused because for example I have ...
0
votes
0answers
14 views

Measurement error in one regressor amongst k regressors

Above I have set out the assumptions and a true claim made by Wooldridge (2002). I am having trouble proving that claim and was wondering if anyone could provide a step-by-step solution of the claim. ...
0
votes
0answers
21 views

In LLSE $e_i$ are orthogonal random variables

If we have a sequence of zero-mean random variables $(y_0,y_1,...,y_{i-1})$ and $$e_i =y_i-\hat{y}_{i|i-1},\ \ \&\ \ \ \hat{y}_{0|-1}=0 $$ $\hat{y}_{i|i-1}=$ LLSE (Linear least square estimate) ...
0
votes
2answers
24 views

Linear Least square estimate of $x^3$ given $x$ and the moments

I have been struggling to find a direction on how to proceed with the following problem. Given that $x$ is a zero mean (non-Gaussian) random variable with moments E$(x^n)=\mu_n$. I need to find the ...
0
votes
1answer
29 views

Calculation of intercept in multiple linear regression (OLS)

While researching OLS, I found out the equation to calculate coefficients as: $$ \beta = (X^\top X)^{-1}X^\top y $$ (Ref: https://en.wikipedia.org/wiki/Linear_least_squares) However it does not ...
1
vote
1answer
45 views

Multicollinearity confusion

so for my master's thesis, I am examining the influence of union density (% of the workforce in a union) and top marginal tax rates on pre-tax CEO pay. These two independent variables are very highly ...
0
votes
0answers
22 views

Least-squares second coefficient meaning

If I have a function $d = \eta D$ where $D$ adn $d$ are obtained experimentally, and I need to calculate $\eta$, the natural thing to do is a least-squares fit ($y=ax+b$). What does the term $b$ mean??...
3
votes
1answer
102 views

How do we know $X'X$ is nonsingular in OLS?

I am currently working through understanding the mechanics of OLS estimates and the hat matrix. One thing I have been searching for without luck is how we know that the term $X'X$ is invertible where $...
0
votes
1answer
29 views

Choice of deflator in OLS

I am running a pooled OLS model and am not able to internalize change in coefficients/ significance due to change in deflator. The OLS specification is as follows: Market Value = Constant + A1*Profit ...
0
votes
0answers
10 views

Individual level DVs with only Group-level IVs?

This is a pooled cross-sectional design where the IVs are at the district-level and the DV is at the individual-level. I also use district- and time-dummies to control for district-specific ...
5
votes
0answers
36 views

Is a visual estimate of homoscedasticity rigorous enough?

As part of my research in astronomy (quasar magnitudes at various wavelengths), I've been producing graphs such as the following: The bottom plot on each graph shows the distribution of the residuals ...
1
vote
1answer
48 views

What is the relationship of long and short regression when we have an intercept?

Consider the linear model estimated by OLS: $$ y = X\hat{\beta} + \hat{u} = X_1 \hat{\beta}_1 + X_2 \hat{\beta}_2 + \hat{u} $$ We say that the above equation is the long regression, Consider also ...
0
votes
1answer
30 views

Why is it unacceptable to use binary or count dependent variables in OLS?

I know this is a basic question but I really want to make sure I fully understand the reason's why this is the case. If possible, can someone help explain to me as simply as possible why it is bad to ...
0
votes
0answers
7 views

Adding insignificant IVs turns another significant predictor insignificant [duplicate]

I have two OLS models: One with three dummy variables and a constant (that represents the fourth dummy). Here, the constant is significantly different from zero at the 10% level. In the second model,...
1
vote
1answer
64 views

when is it possible to have OLS fits better than random forest and LASSO?

I ran several different models on a mini data set of about 100 observations with 90 features. When I tried OLS with backward selection the model is significant with many features significant (82 ...
8
votes
3answers
269 views

Intuition behind $(X^TX)^{-1}$ in closed form of w in Linear Regression

The closed form of w in Linear regression can be written as $\hat{w}=(X^TX)^{-1}X^Ty$ How can we intuitively explain the role of $(X^TX)^{-1}$ in this equation?
0
votes
0answers
15 views

binary least square classification and labelling

I'm trying to do least square method on a set $X \in \mathbb{R}^{100 \times (2+1)}$ (the $+1$ is for the dummy bias feature) for a classification task on 2 classes (NB: no multiclass) and I found that ...
1
vote
1answer
42 views

Relationship eigenvalues of $X'X$ and $(X'X)^{-1}$

I've a question regarding to eigenvalues, sinve I am not very familiar with the concept. Suppose I've a matrix $X'X$ in the case of an OLS regession. And lets assume that the regarding eigenvalues are ...
0
votes
0answers
11 views

Difference between log and growth for interpretation of variable coefficient

I have a question concerning the interpretation of a variable. First of all we know that the proces generating the data is basically: Total = 1*A + 1*B + 1*C + measurment error (So the coefficients ...
0
votes
0answers
56 views

Constrained Linear Regression with coefficients related by inequality

I have a (slightly simplified) model of the following form: $Y=c_1X_1 + c_2X_2 + \epsilon$ subject to the constraint $0\leq c_1\leq c_2$. The distribution of $\epsilon$ is actually not important to ...