18
$\begingroup$

Context:

In Gelman's 8-school example (Bayesian Data Analysis, 3rd edition, Ch 5.5) there are eight parallel experiments in 8 schools testing the effect of coaching. Each experiment yields an estimate for the effectiveness of coaching and the associated standard error.

The authors then build a hierarchical model for the 8 data points of coaching effect as follows:

$$ y_i \sim N(\theta_i, se_i) \\ \theta_i \sim N(\mu, \tau) $$

Question In this model, they assume that $se_i$ is known. I do not understand this assumption -- if we feel that we have to model $\theta_i$, why don't we do the same for $se_i$?

I've checked the Rubin's original paper introducing the 8 school example, and there too the author says that (p 382):

the assumption of normality and known standard error is made routinely when we summarize a study by an estimated effect and its standard error, and we will not question its use here.

To summarize, why don't we model $se_i$? Why do we treat it as known?

Improve this question
$\endgroup$
3
  • $\begingroup$ I assume because they know the total number of schools in the area, so the SE is a function of the sample size and the estimate? $\endgroup$
    – Estimate the estimators
    Commented Jan 8, 2019 at 22:08
  • 1
    $\begingroup$ The sample size is known and fixed, but the standard error depends on the standard deviation of the data as well, and I'm not sure why we're treating that as fixed. $\endgroup$
    – Heisenberg
    Commented Feb 5, 2019 at 23:45
  • 1
    $\begingroup$ If you are happy making your results fully conditional on the assumption of fixed standard errors, then there is nothing wrong with making (and stating) that condition. Still, why? Absence of a defensible prior? Or perhaps if the standard errors are given a broad, uninformative prior, the rest of the analysis just washes away. I dunno. $\endgroup$
    – Peter Leopold
    Commented Feb 6, 2019 at 0:15

1 Answer 1

Reset to default
3
$\begingroup$

On p114 of the same book you cite: "The problem of estimating a set of means with unknown variances will require some additional computational methods, presented in sections 11.6 and 13.6". So it is for simplicity; the equations in your chapter work out in a closed-form way, whereas if you model the variances, they do not, and you need MCMC techniques from the later chapters.

In the school example, they rely on large sample size to assume that the variances are known "for all practical purposes" (p119), and I expect they estimate them using $\frac{1}{n-1} \sum (x_i - \overline{x})^2$ and then pretend those are the exact known values.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I see -- they assume that the variance is estimated very precisely, in other words, that the standard error of the variance is very small? $\endgroup$
    – Heisenberg
    Commented Apr 26, 2019 at 18:27
  • $\begingroup$ Sure. Estimators tend to get more precise with increasing $n$ and the variance is no exception; because $\hat{\sigma}^2$ is a chi-square (times some constant), it has standard deviation $\sqrt{2 \sigma^4 /(n-1)}$. A denominator of about 30 schools seems like it was enough for them. $\endgroup$
    – Drew N
    Commented Apr 26, 2019 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.