# Interpretation of Bayesian 95% prediction interval

Assume the following bivariate regression model: $$y_i = \beta x_i + u_i,$$ where $u_i$ is i.i.d $N(0, \sigma^2 = 9)$ for $i = 1,\ldots, n$.

Assume a noninformative prior $p(\beta) \propto \text{constant}$, then it can be shown that the posterior pdf for $\beta$ is $$p(\beta|\mathbf{y}) = (18\pi)^{-\frac{1}{2}}\left(\sum_{i=1}^n x_i^2\right)^{\frac{1}{2}} \exp\left[-\frac{1}{18}\sum_{i=1}^n x_i^2 (\beta - \hat{\beta})^2\right],$$ where $\hat{\beta} = (\sum_{i=1}^n y_ix_i)/(\sum_{i=1}^n x_i^2).$

Now consider the value of $y$ with a given future value of $x$, $x_{n+1}$: $$y_{n+1} = \beta x_{n+1} + u_{n+1},$$ where $u_{n+1}$ is i.i.d $N(0, \sigma^2 = 9)$, then we can show that $$p(y_{n+1}|x_{n+1},\mathbf{y}) = \int_{\beta} p(y_{n+1}|x_{n+1}, \beta, \mathbf{y}) p(\beta|\mathbf{y})d\beta$$ is a normal density with expectation and variance $$E[y_{n+1}|x_{n+1},\mathbf{y}] = \hat{\beta}x_{n+1},\quad {\rm var}[y_{n+1}|x_{n+1},\mathbf{y}] = \frac{9[x_{n+1}^2 + \sum_{i=1}^n x_i^2]}{\sum_{i=1}^n x_i^2}.$$ Thus the posterior probability density function for $y_{n+1}$, conditional on $x_{n+1}$, is \begin{align} p(y_{n+1}|x_{n+1},\mathbf{y}) = &\left(\frac{18\pi\left[x_{n+1}^2 + \displaystyle{\sum_{i=1}^n x_i^2}\right]}{ \displaystyle{\sum_{i=1}^n x_i^2}}\right)^{-\frac{1}{2}} \\ & \times \exp\left\{-\frac{\displaystyle{\sum_{i=1}^n x_i^2}}{18\left(x_{n+1}^2 + \displaystyle{\sum_{i=1}^n x_i^2}\right)}\left(y_{n+1}-\hat{\beta}x_{n+1}\right)^2\right\} \end{align}

Now the question is: Specify a 95% prediction interval for $y_{n+1}$ and carefully interpret it. What aspect(s) of the data generating process does the interval fail to accommodate our uncertainty about?

I'm not exactly too sure on how to answer the question but here's my attempt:

So essentially we need to find some $a$ and $b$ such that $P(a < y_{n+1} < b) = \int_{a}^b p(y_{n+1}|x_{n+1}, \mathbf{y}) dy_{n+1} = 95\%$

Now we know that $y_{n+1}|x_{n+1}, \mathbf{y} \sim N(m, v^2)$ where $m = E[y_{n+1}|x_{n+1},\mathbf{y}]$ and $v^2 = var[y_{n+1}|x_{n+1},\mathbf{y}]$, Hence: $$\frac{y_{n+1}-m}{v} \sim N(0,1)$$ $$P(-1.96 < \frac{y_{n+1}-m}{v} < 1.96) = 95\%$$ $$P(-1.96v+m < y_{n+1} < 1.96v+m) = 95\%$$

Now because we are conditioning on $x_{n+1}$ and looking at the expression for $v$ and $m$, we see that both $v$ and $m$ are known values. So we can take $a = -1.96v+m$ and $b = 1.96v+m$. i.e., we can select many other possibilities of $a$ and $b$ which yield a probability of $95\%$... but how does this relate to answering the part of the question that asks what aspects of the data generating process this interval fails to accommodate?

• Could it be that the interval specifically doesn't give us any understanding about the shape of data generating process? That is, we don't know the mean/variance combination just from the interval?
– user44764
May 22, 2014 at 0:03
• Strange question. Is there a context prior to the exercise ? Why do you say a bivariate regression model ? Oct 4, 2014 at 20:01

The interval accommodates all of the uncertainty in the problem. In your description of the problem, the only things you don't know are $\beta$ and $u$. The prediction interval that you derived accommodates the uncertainty in both of these. So there is no uncertainty left for the interval to "fail to accommodate".