Assume the following bivariate regression model: $$ y_i = \beta x_i + u_i, $$ where $u_i$ is i.i.d $N(0, \sigma^2 = 9)$ for $i = 1,\ldots, n$.

Assume a noninformative prior $p(\beta) \propto \text{constant}$, then it can be shown that the posterior pdf for $\beta$ is $$ p(\beta|\mathbf{y}) = (18\pi)^{-\frac{1}{2}}\left(\sum_{i=1}^n x_i^2\right)^{\frac{1}{2}} \exp\left[-\frac{1}{18}\sum_{i=1}^n x_i^2 (\beta - \hat{\beta})^2\right], $$ where $\hat{\beta} = (\sum_{i=1}^n y_ix_i)/(\sum_{i=1}^n x_i^2).$

Now consider the value of $y$ with a given future value of $x$, $x_{n+1}$: $$ y_{n+1} = \beta x_{n+1} + u_{n+1}, $$ where $u_{n+1}$ is i.i.d $N(0, \sigma^2 = 9)$, then we can show that $$ p(y_{n+1}|x_{n+1},\mathbf{y}) = \int_{\beta} p(y_{n+1}|x_{n+1}, \beta, \mathbf{y}) p(\beta|\mathbf{y})d\beta $$ is a normal density with expectation and variance $$ E[y_{n+1}|x_{n+1},\mathbf{y}] = \hat{\beta}x_{n+1},\quad {\rm var}[y_{n+1}|x_{n+1},\mathbf{y}] = \frac{9[x_{n+1}^2 + \sum_{i=1}^n x_i^2]}{\sum_{i=1}^n x_i^2}. $$ Thus the posterior probability density function for $y_{n+1}$, conditional on $x_{n+1}$, is \begin{align} p(y_{n+1}|x_{n+1},\mathbf{y}) = &\left(\frac{18\pi\left[x_{n+1}^2 + \displaystyle{\sum_{i=1}^n x_i^2}\right]}{ \displaystyle{\sum_{i=1}^n x_i^2}}\right)^{-\frac{1}{2}} \\ & \times \exp\left\{-\frac{\displaystyle{\sum_{i=1}^n x_i^2}}{18\left(x_{n+1}^2 + \displaystyle{\sum_{i=1}^n x_i^2}\right)}\left(y_{n+1}-\hat{\beta}x_{n+1}\right)^2\right\} \end{align}

Now the question is: Specify a 95% prediction interval for $y_{n+1}$ and carefully interpret it. What aspect(s) of the data generating process does the interval fail to accommodate our uncertainty about?

I'm not exactly too sure on how to answer the question but here's my attempt:

So essentially we need to find some $a$ and $b$ such that $P(a < y_{n+1} < b) = \int_{a}^b p(y_{n+1}|x_{n+1}, \mathbf{y}) dy_{n+1} = 95\%$

Now we know that $y_{n+1}|x_{n+1}, \mathbf{y} \sim N(m, v^2)$ where $m = E[y_{n+1}|x_{n+1},\mathbf{y}]$ and $v^2 = var[y_{n+1}|x_{n+1},\mathbf{y}]$, Hence: $$\frac{y_{n+1}-m}{v} \sim N(0,1)$$ $$P(-1.96 < \frac{y_{n+1}-m}{v} < 1.96) = 95\%$$ $$P(-1.96v+m < y_{n+1} < 1.96v+m) = 95\%$$

Now because we are conditioning on $x_{n+1}$ and looking at the expression for $v$ and $m$, we see that both $v$ and $m$ are known values. So we can take $a = -1.96v+m$ and $b = 1.96v+m$. i.e., we can select many other possibilities of $a$ and $b$ which yield a probability of $95\%$... but how does this relate to answering the part of the question that asks what aspects of the data generating process this interval fails to accommodate?

  • $\begingroup$ Could it be that the interval specifically doesn't give us any understanding about the shape of data generating process? That is, we don't know the mean/variance combination just from the interval? $\endgroup$
    – user44764
    May 22, 2014 at 0:03
  • $\begingroup$ Strange question. Is there a context prior to the exercise ? Why do you say a bivariate regression model ? $\endgroup$ Oct 4, 2014 at 20:01

1 Answer 1


The interval accommodates all of the uncertainty in the problem. In your description of the problem, the only things you don't know are $\beta$ and $u$. The prediction interval that you derived accommodates the uncertainty in both of these. So there is no uncertainty left for the interval to "fail to accommodate".

  • $\begingroup$ There remains uncertainty as to whether or not the model is correct. $\endgroup$
    – Xi'an
    Nov 9, 2014 at 16:08
  • 1
    $\begingroup$ The model was stated as an assumption. Therefore there is no uncertainty about it. $\endgroup$
    – Tom Minka
    Nov 9, 2014 at 22:30
  • $\begingroup$ I suppose what is meant is that the predictive distribution represents all the uncertainty in the problem and the interval only summarises some aspect of that distribution. Still, it's not clear that as a interval it leaves anything out. $\endgroup$ Dec 8, 2014 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.