0
$\begingroup$

I have found some information concerning my problem, but I am too unfamiliar with statistics to fully grasp the concepts explained. Also, much of the explanations are written for python/Matlab/R code, which I do not need. I hope you can help me out.

I am trying to answer a question from an exercise on multivariable normal distributions. The question is: based on the following expectation vectors and their corresponding covariance matrices, sketch (on paper!) the scatter plots that would appear if you'd take a large number of samples from the distributions. Given are:

$\mu_1 = \begin{pmatrix}0 \\ 0 \end{pmatrix}, \Sigma_1 = \begin{pmatrix} 1 & 0 \\ 0 & 1\end{pmatrix}$

$\mu_2 = \begin{pmatrix}3 \\ 2 \end{pmatrix}, \Sigma_2 = \begin{pmatrix} 4 & 0 \\ 0 & 1\end{pmatrix}$

$\mu_3 = \begin{pmatrix}3 \\ 2 \end{pmatrix}, \Sigma_3 = \frac{1}{\sqrt{10}}\begin{pmatrix} 1 & -3 \\ 3 & 1\end{pmatrix}\begin{pmatrix} 4 & 0 \\ 0 & 1\end{pmatrix}\frac{1}{\sqrt{10}}\begin{pmatrix} 1 & -3 \\ 3 & 1\end{pmatrix}^T$

I am at a loss how to start with the first one, which should be the easiest. However, I do recognise that the third covariance matrix is written in the form $UDU^T$. I do not wish the answer to this question, as I try to learn how to do it. Some help, though, would be much appreciated!

$\endgroup$

1 Answer 1

0
$\begingroup$

First you must understand the meaning of each entry in the mean vector and covariance matrix. Say, in case 1, the means are zero and the variables are uncorrelated (and independent as they are Gaussian). So, when you plot the variables on an X-Y axis (where X corresponds to the first variable and Y to the second), the x value will be determined independently of the second. This is what the off diagonals tell you, and they are zero.

So you will draw (no pun intended) samples from a zero-mean distribution and then you'll have your x value for the scatter plot, and you'll determine the y value similarly. Obviously, you see that if, for instance, your point has a high x value, it has no affect on the y value - it can be high, low, close to zero, etc.

Furthermore, we need to look at the diagonal value. In case 1, the variances of the variables are equal. Meaning, the effective 'width' or spread of the points on paper will be the same - so that if you, say, rotate the paper by 90 degrees, you'll see the same shape.

Now, what would happen if the means aren't zero? this is rather simple - the center of mass of your points will shift from the center to the mean, for each variable. Slightly more difficult is what happens if the variances are not equal. In this case, you'll need to draw the variable with the higher variance in a more 'spread' manner, regardless of its dependence in the other variable.

Finally, the most difficult part is the off diagonals in the covariance matrix - Say you have positive covariance between X and Y. Then, a high x value will likely go with a high y value, and vice versa - if they have negative correlation, then they'll tend to have values on opposite sides w.r.t the means (if the means are zero, negative correlation implies that when x is positive, y will tend to be negative, etc.).

$\endgroup$
4
  • $\begingroup$ Okay, thanks! I now understand that the expectation vector determines the center of my plot (in case of the first (0,0)), so that's a start. I also understand from your answer that the shape is dependent on the covariance matrix: if the matrix has the same values, it's a circle, right? However, how do I determine the radius of the circle? $\endgroup$
    – Nifty
    Commented May 23, 2016 at 10:37
  • $\begingroup$ This is what I meant by width or spread - the diagonals in the covariance determine the radius. If they're equal then it's a circle, and if they're unequal then it's an ellipse. $\endgroup$
    – yoki
    Commented May 23, 2016 at 10:58
  • $\begingroup$ Right! So I figured the first one should be a circle with radius 3, the second one an ellipse with axis 3 and 6 (because that would be determined by the standarddeviation i.e sqrt(variance)). However, the last one is a bit tricky now.. what does the 1/sqrt(10) mean, and how to I draw the ellipse? (thank you for helping me out, anyway!) $\endgroup$
    – Nifty
    Commented May 23, 2016 at 11:01
  • $\begingroup$ One of the ways to interpret this is to calculate the matrix multiplication and then use the covariance matrix you obtain to draw what will be the rotated elipse describing the distribution. $\endgroup$
    – yoki
    Commented May 23, 2016 at 11:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.