1
$\begingroup$

this is my first post!

I know the sampling distribution of the sample mean is $N(\mu,\frac{\sigma^2}{n})$.

I also know that the sampling distribution of $\hat \beta_1$ is $N(\beta_1, \frac{\sigma^2}{\sum(x-\bar x)^2})$

However, this last variance depends on the specific values for a given sample! How can that be? I have run a simulation in R just to check.

Here's a demonstration. I am creating 1000 data points with $\beta_1=2$ and $\sigma^2=1$. I then take 1000 samples of size 50, estimate $\hat \beta_1$, and then take the standard deviation of the $\hat \beta_1$s. When I do, I find that it is

x<-seq(-5,4.99,.01)
y<-1+2*x+rnorm(1000,mean=0,sd=1)
dat<-data.frame(x,y)

betas<-vector()
sigma2<-vector()

for(i in 1:1000){
sum1<-summary(lm(data=dat[sample(nrow(dat),size = 50,replace = F),],y~x))
sum1
betas<-rbind(betas,coef(sum1)[1:2,1])
sigma2<-rbind(sigma2,sum1$sigma^2)
}

mean(betas[,2])
#1.999
sd(betas[,2])
#.0505

When I take a specific sample of 50 and compute the standard deviation myself, I get near the correct value

dat1<-dat[sample(nrow(dat),size = 50,replace = F),]
with(dat1,1/sqrt(sum((x-mean(x))^2)))
# 0.05027655

I'm just confused about how the population variance of the sampling distribution can be based on the actual values of $x$ in a given sample.

$\endgroup$
  • 2
    $\begingroup$ In regression, $x$ is treated as known, not random. More precisely, inference is conditional on the design matrix, $X$ $\endgroup$ – Glen_b May 6 '18 at 0:37
  • $\begingroup$ I just wanted to comment that the two claims you make require that in 1) the samples are independent and identically distributed and normal with mean $\mu$ and variance $\sigma^2$ and in 2) the error terms are independent and normally distributed with mean 0 and constant variance. $\endgroup$ – Michael Chernick May 6 '18 at 2:30
2
$\begingroup$

In the case where your predictors are nonrandom, this should be intuitive because

  1. the more data you have, the smaller the variance, and
  2. the more dispersed the $x$ data are, the smaller the variance.

I suspect you are worried about the duality of sample and population. You're probably saying to yourself, data cannot be a parameter because it is random. And that's still appropriate to stick with that intuition here. The assumptions you are using, however, say that $x_1, \ldots, x_n$ are nonrandom. They are not a random sample.

$\endgroup$
  • $\begingroup$ The x values are selected by design. Taylor's comments 1 and 2 are very true. Note that if all the xs are close together the slope is not very accurate. If you take 1 point far away from a set of closely together xs could lead t a big change in th estimate of slope. $\endgroup$ – Michael Chernick May 6 '18 at 2:22
  • $\begingroup$ Thanks Taylor and @Glen_b. I think I understand. So there is no true "population" value for for the standard deviation of the sampling distribution of a regression coefficient (that is sample independent)? Like, there is no analytic expression for the true value? I guess that just seems counter-intuitive because the sampling distribution of the mean is so easy, sigma^2/sqrt(n). But if I repeatedly estimate new regression coefficients based on samples from the population, can I think of the standard deviation of this empirical sampling distribution as the "true" sd of the sampling distribution? $\endgroup$ – nrw May 9 '18 at 0:26
  • $\begingroup$ @nrw "So there is no true "population" value for for the standard deviation of the sampling distribution of a regression coefficient (that is sample independent)?" <- It's $\sigma/\sqrt{\sum_i (x_i - \bar{x})^2}$. "But if I repeatedly estimate new regression coefficients based on samples from the population, can I think of the standard deviation of this empirical sampling distribution as the "true" sd of the sampling distribution?" <- You'll have to be more precise. Perhaps consider asking another question on the site. $\endgroup$ – Taylor May 11 '18 at 4:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.