# What is the simple standard error for MCMC?

Simply put: suppose that we have observed $X=\left\{ X_{1},\ldots,X_{n}\right\}$. We then need to calculate some statistic $T$ using MCMC, using $M$ loops (By "loops" I mean the number of times the chain is repeated to come up with a sensible posterior). We then repeat this entire experiment $P$ times as a part of some sort of simulation study.

What is a sensible estimate of the standard error of $T$?

At the moment, I am calculating the standard error without considering $n$ - so I do not think this is correct.

• Do you have any burn-in (/warm-up)? Are you skipping or using all values in the sequence? – Glen_b Jan 11 '15 at 12:02
• If you repeat the experiment $P$ times, this is an iid experiment, so you can use standard methods to estimate the average over the $P$ repetitions. I do not think $n$ should appear explicitly in the standard error formula. – Xi'an Jan 11 '15 at 12:34
• I think you can just use the sample standard deviation. That's what I do – shadowtalker Jan 11 '15 at 13:40
• Glen - I am using a burn-in period, but I assumed it is 0 for simplicity Xi'an - if this is the case, then surely there would be more information using 10,000 actual observations Vs. 10? – akkp Jan 12 '15 at 4:29

I know very little frequentist stats so I can't say anything about the standard error, but I have come across an estimate of the variance that might help.

Suppose after burnin you have $C$ chains of $S$ samples each. Then define the between-chain variance as

$$B = \frac{S}{C-1} \sum_c (\bar y_c - \bar y)^2$$

where $\bar y_c$ is the mean of chain $c$ and $\bar y$ is the overall mean. Define also the within-chain variance

$$W = \frac{1}{C} \sum_c \frac{1}{S-1} \sum_s (y_{cs} - \bar y_c)^2$$

where $y_{sc}$ is the $s$th sample in the $c$th chain. Then under stationarity, an unbiased estimate of the variance of $y$ is

$$\hat V = \frac{S-1}{S}W + \frac{1}{S}B$$

This is originally from Gelman & Rubin 1992, Section 2.2, though this particular notation is from MLAPP section 24.4.3.1.

• Note that this is just the weighted average of sample variances – shadowtalker Jan 11 '15 at 13:40
• Thanks Andy. How about the number of observations which the MCMC used to come up with $y_{i}$? – akkp Jan 12 '15 at 4:34
• What do you mean? – Andy Jones Jan 12 '15 at 14:08
• For example, we simulate 100 observations from some $i.i.d N(0,1)$, and we wish to calculate the AR(1) coefficient using MCMC (obviously this is going to be 0, but we want MCMC to prove it). So we use a MCMC sampler with 10,000 loops (0 burn-in for simplicity) and have 500 independant chains running at once. How does the 100 come into play here when calculating the s.e. of the AR(1) coefficient? – akkp Jan 15 '15 at 21:19
• Note: I am starting to think this is "automagically" taken into consideration, and there is no need to adjust for this number – akkp Jan 18 '15 at 12:43