2
$\begingroup$

I would like to show that

$$E(\bar{x} \hat{\beta})=E(\bar{x}) E(\hat{\beta})$$

and

$$E(\bar{y} \hat{\beta})=E(\bar{y}) E(\hat{\beta})$$

where in the population it holds that

$$y_i=\alpha + \beta x_i+\epsilon_i$$ where $\epsilon_i$ are independetly, normally distributed errors with $N(0,\sigma^2_{\epsilon})$ and $\hat{\beta}$ is the OLS esitmator, and $\bar{x}$ and $\bar{y}$ sample means respectively.

In words, I would like to show that sample means and OLS regression coefficient estimates are independent random variables. Can you provide guidance on how to approach the problem?

Edit I ran a simulation and it stresses the independence assumption. But it's contrary to @kjetil_b_halvorsen reply.

n=10000
boot=10000

xbar=numeric(); ybar=numeric(); b=numeric()
for(i in 1:boot){
x=rnorm(n,10,1)
y=1+2*x+rnorm(n)
xbar[i]=mean(x)
ybar[i]=mean(y)
b[i]=lm(y~x)$coefficients[2]
}

> mean(xbar*b)
[1] 20.0009
> mean(xbar)*mean(b)
[1] 20.0009
> 
> mean(ybar*b)
[1] 42.0018
> mean(ybar)*mean(b)
[1] 42.0018
$\endgroup$
7
  • 1
    $\begingroup$ Do you really mean $iid(\mu_\epsilon,\sigma²_\epsilon$ without any specification of distribution form, not, for example, normal distribition? If so, this is impossible. $\endgroup$ – kjetil b halvorsen Sep 8 '15 at 12:23
  • $\begingroup$ I made an error; it should say zero mean errors. I think then it's solvable. $\endgroup$ – tomka Sep 8 '15 at 12:30
  • $\begingroup$ OK, but zero mean is still not enough, it could for instance be true if normal distribution but not in other cases. $\endgroup$ – kjetil b halvorsen Sep 8 '15 at 12:35
  • $\begingroup$ Really? I did not expect that. I change to normal distributed error terms. For starters. $\endgroup$ – tomka Sep 8 '15 at 12:38
  • 1
    $\begingroup$ When the $(x_i,\varepsilon_i)$ do not have a bivariate normal distribution, it's unlikely $\bar y$ and $\hat \beta$ will be independent. Take a look: n <- 8; sim <- replicate(1e4, { x <- rexp(n); z <- x - mean(x); y <- x + rnorm(n); beta.hat <- sum(y * z) / sum(z * z); y.bar <- mean(y); c(y.bar=y.bar, beta.hat=beta.hat, product=y.bar * beta.hat) }) plot(sim["y.bar", ], sim["beta.hat", ], pch=16, cex=0.5, col="Gray") $\endgroup$ – whuber Sep 8 '15 at 21:41
1
$\begingroup$

In the normal distribution case, independence is tha same as covariance zero. So it is enough to calculate the covariance. Doing that, you will find the covariance between $\hat{\beta}$ and $\bar{x}$ is given by $$ \text{some constant} \cdot \sum_{i=1}^n x_i $$ so can be zero only if that last sum is zero. So, in that case, you have independence (in the normal case, without normal assumption you can only conclude the covariance is zero).

$\endgroup$
6
  • $\begingroup$ Please see edit on top. $\endgroup$ – tomka Sep 8 '15 at 20:42
  • $\begingroup$ In your simulation you include an intercept, which is not in your stated model! So ir is not the same model. Which do you mean, which is your intended model? $\endgroup$ – kjetil b halvorsen Sep 8 '15 at 20:54
  • $\begingroup$ The one with intercept. $\endgroup$ – tomka Sep 8 '15 at 21:10
  • $\begingroup$ So edit the original post to clarify. My answer is for the model you stated first, without intercept. $\endgroup$ – kjetil b halvorsen Sep 8 '15 at 21:13
  • $\begingroup$ Edited - sorry for the confusion $\endgroup$ – tomka Sep 9 '15 at 7:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.