# Constructing an interval estimate for a multivariate output

I am building a multivariate Gaussian Process model that predicts output $X_i$ and $Y_i$ jointly from input location $z_i$. At the end of it all I want to be able to plot my best estimate of $X_i$ against $Y_i$ where I know that

$$\begin{pmatrix} X_i\\ Y_i \end{pmatrix} \sim N_2 \begin{pmatrix} \begin{pmatrix} \mu_{X_i}\\ \mu_{Y_i} \end{pmatrix}, \begin{pmatrix} \sigma_{11i} & \sigma_{12i}\\ \sigma_{21i} & \sigma_{22i}\\ \end{pmatrix} \end{pmatrix}$$ So as you can see, each $X_i,Y_i$ pair has their own mean and covariance matrix. Now, in order to plot my best estimate of $X_i$ against $Y_i$ I can simply plot my my predicted values of $X_i$ against $Y_i$ (the red line shown below). Now what my question is whether or not it is appropriate (or if there is a better way) to construct a 95% interval estimate in the following fashion. So for each point $X_i,Y_i$ I can plots its 95% confidence ellipse as seen below (the blue ellipse)

And for another point I can do similarly.

And for a third point I can do the same again where we see that there is a very apparent difference in covariance matrix here for this set of $X_i$ and $Y_i$

And so finally we see that we could plot a 95% interval estimate for every single point $X_i$ and $Y_i$ (the last two plots, where the second to last plot has the same covariance matrix for all the points and the last plot has varying covariance matrix for all the points). Now, can I consider this a 95% joint interval for all points $X_i$ and $Y_i$ or is there a better way to go about this?

• One always wonders what constitutes an official source for statistical knowledge ... – Glen_b Sep 9 '13 at 6:11
• honestly I did not know what better way to tag the question. Really I will award the points to simply the best answer. – user25658 Sep 9 '13 at 13:27
• My comment wasn't intended as suggesting you had done anything wrong at all with that choice. It's one of those options that fits well enough with SO (since for some programming issues there kind of are 'official' answers for specific products), but the wording sounds odd here on CV. If you're looking for something better substantiated than just idle opinion, that's the only option you have like it. – Glen_b Sep 9 '13 at 23:01

[I know this doesn't answer the question, nor provide sources; this was going to be a comment but got too long]

I think your problem here is you haven't clearly defined "95% confidence interval", and your problem presents with more than one way of interpreting that. If you decide exactly what you mean by "95% confidence interval" you will probably answer your own question.

For example, do you mean:

1. If I reran my experiment many times, 95% of the regions so generated would completely contain the true path

2. If I reran my experiment many times, the regions would on average contain 95% of the true points

3. If I reran my experiment many times, then for any given true $X_i$, the corresponding true $Y_i$ would be contained in the vertical region above $X_i$ region 95% of the time.
4. If I reran my experiment many times, then for any given true $Y_i$, the corresponding true $X_i$ would be contained in the horizontal region across from $Y_i$ region 95% of the time.
5. A region containing 95% of the posterior probability density for a point $i$ picked at random. This is probably closest to what you have generated.

Or perhaps some other variation.

Your problem is that your estimate is strictly a very high dimensional vector of all the $(X_i, Y_i)$ pairs, and so your "standard" confidence region would be a high dimensional ellipsoid. Remember that the uncertainty around each point in the process need not be independent of the previous points, if one step is way too high, the next will probably be as well. Because this is a Gaussian process, you should be able to work out the full 2N x 2N correlation matrix, and so you can work a lot of different confidence regions depending on what interests you.