# How do I determine the distribution of an additional sample $x_{n+1}$ given sample mean, variance $\bar x, s^2$ of a normally distributed $x$?

Suppose I draw $$n$$ samples $$x_1...x_n$$ of a random variable $$x$$, which is normally distributed with an unknown mean $$\mu$$ and variance $$\sigma^2$$. From those samples, I compute a sample mean $$\bar x$$ and a sample variance $$s^2$$. I wish to compute the distribution of $$x_{n+1} \mid \bar x, s^2$$, i.e. the distribution of one additional sample $$x_{n+1}$$, given my measured $$\bar x$$ and $$s^2$$.

How should I proceed? This is probably a bad way of proceeding with loads of issues, but here is my thought process so far:

1. Start with the fact that $$\frac{\bar{x} - \mu}{s/\sqrt{n}}$$ is $$t$$-distributed, and then use that to find a likelihood of $$\mu$$ given $$\bar x$$.
2. Use the fact that $$\frac{x_{n+1} - \mu}{\sigma} \sim \mathcal N(0,1)$$.
3. Use the fact that $$\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$$
4. Combine these facts to get a distribution of $$x_{n+1}$$.

An alternative approach I thought of goes as follows:

1. $$x_{n+1}\sim N(\mu, \sigma^2)$$
2. $$\bar x \sim N(\mu, \frac{\sigma^2}{n})$$
3. $$(x_{n+1} - \bar x) \sim N(0, \sigma^2 + \frac{\sigma^2}{n})$$
4. Find the distribution of $$\sigma^2 \mid s^2$$ using the fact that $$\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$$
5. Integrate $$N(0, \sigma^2 + \frac{\sigma^2}{n})$$ over all possible $$\sigma^2$$ weighted by the distribution of $$\sigma^2 \mid s^2$$
• Somewhere a dead monk named Bayes is turning in his grave at these hoops the Frequentists have made you jump through. Oct 28, 2021 at 18:55
• The point about Bayes is apt because the question as posed does not have a solution: the conditional distribution depends on the unknown parameters. The classical approach is to base testing or prediction on the estimated joint distribution of $(\bar x, x_{n+1}),$ while the Bayesian approach updates a prior distribution on the parameters--and gives the kind of answer you are looking for.
– whuber
Oct 28, 2021 at 19:07
• oh no its not your fault @Danny, I'm just lamenting the state of statistical education Oct 28, 2021 at 19:13
• You don't have "samples" $x_1,\ldots,x_n$; rather you have a sample $x_1,\ldots,x_n.$ And $x_{n+1}$ is not one additional sample; it is one additional observation. Oct 28, 2021 at 20:24
• The word sample has a particular meaning in statistics. You claimed to be a lowly physicist but then attempt to correct a statistician on the meaning of the term in this statistical setting.... That's a risky strategy. en.wikipedia.org/wiki/Sample_(statistics) Oct 29, 2021 at 1:37

The distribution of $$x_{n+1}$$ conditional on $$\overline x=(x_1 +\cdots +x_n)/n$$ and $$s^2 = \big((x_1-\overline x)^2+\cdots+(x_n-\overline x)^2\big)/(n-1)$$ is $$\operatorname N(\mu,\sigma^2),$$ since the observations $$x_1,\ldots,x_n,x_{n+1}$$ are independent. Thus I suspect that what you actually want is the distribution of $$(x_{n+1}-\overline x)/s,$$ upon which you can base a prediction interval whose endpoints are $$\overline x\pm c\cdot s_n,$$ where $$c$$ is chosen so as to get a desired probability that $$x_{n+1}$$ is in the interval.

Notice that $$x_{n+1}-\overline x\sim\operatorname N\left(0,\sigma^2\left( 1 + \frac 1 n \right)\right),$$ and since $$s^2$$ is independent of $$\overline x$$ and of $$x_{n+1},$$ and $$(n-1)s^2/\sigma^2\sim\chi^2_{n-1},$$ you have $$\frac{(x_{n+1}-\overline x)/\sqrt{1+\frac 1 n}}{s/\sqrt n} \sim t_{n-1}.$$ Thus if you choose $$c$$ so that $$\Pr(-c then you have $$\Pr\left(x_{n+1} \text{ is between } \overline x \pm c\cdot\frac s {\sqrt n}\cdot\sqrt{1+\tfrac 1 n} \right) = 1-\alpha.$$ This is a prediction interval for $$x_{n+1}.$$

• I think precisely, I want the distribution of $x_{n+1} - \overline x$ in terms of $s$ rather than $\sigma$. Maybe I'm thinking about it incorrectly, but either way, your answer gets me there, along with @Mang's. Oct 28, 2021 at 23:30

As @whuber points out, this question is much more naturally answered in a Bayesian context. In that case the distribution you are interested in is known as the posterior predictive distribution.

$$p(x_{n+1}|x_1,\ldots,n) = \int_\theta p(x_{n+1}|\theta,x_1,\ldots,x_n)p(\theta|x_1,\ldots,x_n)d\theta$$

where $$\theta = (\mu,\sigma^2)$$. This is about as close as you can get in the general case. If you are willing to accept a particular prior distribution for $$\mu$$ and $$\sigma^2$$ (Normal for $$\mu$$ and Inverse-Gamma for $$\sigma^2$$) you can get a closed form distribution (it is a $$t$$ distribution) which you can look up here

I believe that the following holds:

$$T^*:= \sqrt{\frac{n}{n+1}}\frac{(X_{n+1}-\bar{X_n})}{s_n} \sim t_{n-1}$$

So if you define the test statistic $$T^*$$, you can thus derive confidence intervals, point estimates, and other quantities of interest with respect to $$X_{n+1}$$.

Proof:
First, note that $$X_{n+1}, \bar{X_n}, s_n$$ are all mutually independent.

Then, $$\bar{X_n} \sim N(\mu, \frac{\sigma^2}{n})$$, $$X_{n+1} \sim N(\mu, \sigma^2)$$, which implies (per your footnote) that $$X_{n+1} - \bar{X_n} \sim N(0,\frac{n+1}{n} \sigma^2)$$.

Thus, $$Z:= \frac{1}{\sigma} \sqrt{\frac{n}{n+1}} (X_{n+1} - \bar{X_n}) \sim N(0,1)$$

It's well known that $$V:= (n-1) \frac{s^2_n}{\sigma^2} \sim \chi^2_{n-1}$$ where $$s^2_n = \frac{1}{n-1}\sum_{i=1}^n (X_i-\bar{X})$$.

Since $$Z$$ and $$V$$ are independent, by definition of the t distribution, it follows that:

$$T^* = Z \sqrt{\frac{n-1}{V}} \sim t_{n-1}$$

This can be easily verified by simulation as well: Simulate n+1 independent normals using your favorite software, look at the density plot of your test statistic $$T^*$$ and compare with the $$t_{n-1}$$ distribution.