How to add prediction intervals to describe the variability in monthly samples of data?

Question

The data analyzed here is a sample of individuals collected on a monthly basis. What would be the best way to compute "prediction intervals" for the monthly sample means, in order to indicate the natural variability that can be expected from this value month after month?

Here is an example:

• All individuals are sampled with replacement from the same population whose mean is expected be the same across all months;
• $y_i$ for individual $i$ is the variable of interest, with $ y_i \sim N(y,\sigma)$, both $y$ and $\sigma$ are unknown;
• $N_m$ is the sample size for month $m$;
• $\bar y_m$ is the sample mean for month $m$.

What would be the best way to compute a 95% prediction interval for $\bar y_m$?

I thought of doing the following:

• $\bar y $ is the sample mean across the whole sample;
• $\bar \sigma $ is the standard deviation across the whole sample;
• For each month $j$, I compute the interval as $[ \bar y - 1.96 \frac{\bar \sigma}{ \sqrt{N_m}} ; \bar y + 1.96 \frac{\bar \sigma}{ \sqrt{N_m}} ]$.

I'm afraid this is not statistically correct. Simulations I've been doing show that this interval contains the monthly value more than 95 times out of 100. I assume this is caused by the fact that I do not consider the uncertainty in the estimation of $\bar y$.

What would be the correct way to proceed?

Thanks.

Answer 1

If I understand correctly, in each month, you measure several individuals once. Then you work out the mean and standard deviation of some variable. Without knowing anything else I'd base confidence intervals on the t-distribution; the intervals would be a bit wider than those using a multiplier of 1.96, but it would take a small sample to show much difference.

However, you are saying nothing about what it is that is being measured or counted, and depending on that and what its distribution is like, the quality of a t-distribution approximation might vary a bit.

Also, nothing in your analysis addresses any dependence in time that might exist, but as you are averaging across individuals within months that is not necessarily an issue. You might want to consider whether there are dependencies between individuals that are biting.

All that said, your situation sounds good. You've checked your procedure by simulation and your coverage is a bit more than expected. Few researchers could say as much, and if it were me, I would not go further and try to correct anything. No need to fix what is not evidently broken.