Simply put: suppose that we have observed $X=\left\{ X_{1},\ldots,X_{n}\right\}$. We then need to calculate some statistic $T$ using MCMC, using $M$ loops (By "loops" I mean the number of times the chain is repeated to come up with a sensible posterior). We then repeat this entire experiment $P$ times as a part of some sort of simulation study.

What is a sensible estimate of the standard error of $T$?

At the moment, I am calculating the standard error without considering $n$ - so I do not think this is correct.

Many thanks in advance!

  • $\begingroup$ Do you have any burn-in (/warm-up)? Are you skipping or using all values in the sequence? $\endgroup$ – Glen_b Jan 11 '15 at 12:02
  • $\begingroup$ If you repeat the experiment $P$ times, this is an iid experiment, so you can use standard methods to estimate the average over the $P$ repetitions. I do not think $n$ should appear explicitly in the standard error formula. $\endgroup$ – Xi'an Jan 11 '15 at 12:34
  • $\begingroup$ I think you can just use the sample standard deviation. That's what I do $\endgroup$ – shadowtalker Jan 11 '15 at 13:40
  • $\begingroup$ Glen - I am using a burn-in period, but I assumed it is 0 for simplicity Xi'an - if this is the case, then surely there would be more information using 10,000 actual observations Vs. 10? $\endgroup$ – akkp Jan 12 '15 at 4:29

I know very little frequentist stats so I can't say anything about the standard error, but I have come across an estimate of the variance that might help.

Suppose after burnin you have $C$ chains of $S$ samples each. Then define the between-chain variance as

$$B = \frac{S}{C-1} \sum_c (\bar y_c - \bar y)^2$$

where $\bar y_c$ is the mean of chain $c$ and $\bar y$ is the overall mean. Define also the within-chain variance

$$W = \frac{1}{C} \sum_c \frac{1}{S-1} \sum_s (y_{cs} - \bar y_c)^2$$

where $y_{sc}$ is the $s$th sample in the $c$th chain. Then under stationarity, an unbiased estimate of the variance of $y$ is

$$\hat V = \frac{S-1}{S}W + \frac{1}{S}B$$

This is originally from Gelman & Rubin 1992, Section 2.2, though this particular notation is from MLAPP section

  • 1
    $\begingroup$ Note that this is just the weighted average of sample variances $\endgroup$ – shadowtalker Jan 11 '15 at 13:40
  • $\begingroup$ Thanks Andy. How about the number of observations which the MCMC used to come up with $y_{i}$? $\endgroup$ – akkp Jan 12 '15 at 4:34
  • $\begingroup$ What do you mean? $\endgroup$ – Andy Jones Jan 12 '15 at 14:08
  • $\begingroup$ For example, we simulate 100 observations from some $i.i.d N(0,1)$, and we wish to calculate the AR(1) coefficient using MCMC (obviously this is going to be 0, but we want MCMC to prove it). So we use a MCMC sampler with 10,000 loops (0 burn-in for simplicity) and have 500 independant chains running at once. How does the 100 come into play here when calculating the s.e. of the AR(1) coefficient? $\endgroup$ – akkp Jan 15 '15 at 21:19
  • 1
    $\begingroup$ Note: I am starting to think this is "automagically" taken into consideration, and there is no need to adjust for this number $\endgroup$ – akkp Jan 18 '15 at 12:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.