I have a covariate $B$ (let's say age) and two different responses $T_1$ and $T_2$. The bivariate distributions of $B,T_1$ as well as $B,T_2$ are bivariate normal and known:

$$ \begin{pmatrix}B\\T_1\end{pmatrix} \sim N \left[ \begin{pmatrix}\mu_B\\\mu_1\end{pmatrix} , \begin{pmatrix}\ \sigma_B^2 & \sigma_{1B} \\ \sigma_{1B} & \sigma_1^2 \end{pmatrix} \right] $$


$$ \begin{pmatrix}B\\T_2\end{pmatrix} \sim N \left[ \begin{pmatrix}\mu_B\\\mu_2\end{pmatrix} , \begin{pmatrix}\ \sigma_B^2 & \sigma_{2B} \\ \sigma_{2B} & \sigma_2^2 \end{pmatrix} \right] $$

My goal now is to find the distribution of $\begin{pmatrix}B\\T_1-T_2\end{pmatrix}$.

Here is an image to visualize. The distribution of the black ($T_1$) and red ($T_2$) cloud in the top figure are given, and I am looking for the distribution of the bottom cloud.

multivariate gaussian pic

I have everything except for the covariance between $T_1$ and $T_2$:

  • I know my distribution is a bivariate normal again
  • The mean vector of that distribution is simple: $(\mu_B, \mu_1-\mu_2)^T$
  • From the covariance matrix, the top left element is $\sigma_B^2$, and the two off-diagonal elements are $\sigma_{1B}-\sigma_{2B}$.
  • The variance of $T_1-T_2$ is a problem. I know: $$\text{Var}(T_1-T_2) = \sigma_1^2 + \sigma_2^2 - 2 \cdot \text{Cov}(T_1,T_2)$$

But I have no idea how to compute $\text{Cov}(T_1-T_2)$. I also know:

$$ \text{Cov}(T_1,T_2) = \text{E}(T_1T_2) - \mu_1 \mu_2$$

So equivalently, I am looking for $\text{E}(T_1T_2)$ .

Does anyone have an idea how to get one of these two values?

Edit: Just to clarify: I am not looking for the sample covariance, but for the 'real' covariance. The ovservations in the image are just for visualization

  • 1
    $\begingroup$ For what reason can you not calculate $COV(T_1,T_2)$? Is this output from a model? You will need to estimate the covariance for this problem. $\endgroup$ – Zachary Blumenfeld Feb 12 '16 at 20:33
  • $\begingroup$ Based on your graphs, It looks like you can estimate the covariance using the sample estimate $\endgroup$ – Zachary Blumenfeld Feb 12 '16 at 20:36
  • $\begingroup$ I obtained the samples in the bottom graph through a very inconvenient detour, with kernel density estimates and stuff. I could estimate that sample's covariance, but I would prefer to have the 'true' value, given the known parameters of the other two distributions. $\endgroup$ – Alexander Engelhardt Feb 12 '16 at 20:39
  • $\begingroup$ I don't know what "kernel density estimation and stuff" is (maybe you can provide more context here). I do know that, in general, you cannot estimate/know the $var(T_1-T_2)$ without estimating/knowing the $cov(T_1,T_2)$. $\endgroup$ – Zachary Blumenfeld Feb 12 '16 at 20:45
  • $\begingroup$ Exactly. That's why I am looking for $cov(T_1,T_2)$ here. The 'kernel density estimation' part is, as far as I see it, irrelevant to my question. I just used it to generate the plot. $\endgroup$ – Alexander Engelhardt Feb 12 '16 at 20:50

$cov(T_1,T_2)$ is not uniquely determined by the information you have given.

You have not specified that $B,T_1, T_2$ are 3-dimensional Normal, but I will assume you have intended this. Therefore, you should think of a 3 by 3 covariance matrix for this 3-dimensional Normal, whose only constraints are that the covariance entries you have provided in the 2 by 2 matrices hold, and that the 3 by 3 matrix is (symmetric) positive semidefinite.

As an example, let all 3 component variances = 1, and $cov(B,T_1) = cov(B,T_2) = 0.1$. In this example, as I determined by applying semidefinite optimization, $cov(T_1,T_2)$ can be anywhere in the range [-0.98,1], and the 3 by 3 matrix will indeed be positive semidefinite.

Of course, if you change the data in my above example to other values, the range of possible $cov(T_1,T_2)$ will change accordingly.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.