Why are contours of a multivariate Gaussian distribution elliptical? Displayed below are the contours and their respective covariance matrices according to Andrew Ng's notes (pdf). Why are the first and second contours elliptical and not circular? The variance along both axes is the same.

Here's one last set of examples generated by varying $\Sigma$:  
 
The plots above used, respectively,
  $$
\Sigma = \begin{bmatrix} 1&-0.5\\-0.5 &1 \end{bmatrix}; \qquad 
\Sigma = \begin{bmatrix} 1&-0.8\\-0.8 &1 \end{bmatrix}; \qquad 
\Sigma = \begin{bmatrix} 3&0.8\\0.8 &1 \end{bmatrix}.
$$  

 A: Assume you are visualizing the distribution of a vector called $(X,Y)$ (assumed to have a bivariate normal distribution).
When  $X$ and $Y$ have the same variance, the projections of the ellipse on both axes have the same length. This does mean it's a circle. It can be oblique. It's not a circle when $X$ and $Y$ are not independent.
When $X$ and $Y$ are independent, the major and minor axes of the ellipse are aligned with the axes. This does not mean it's a circle either, it can be flattened.
A circle requires both:


*

*independence of $X$ and $Y$

*$X$ and $Y$ having the same variance


This is when the covariance matrix $\Sigma$ is diagonal with a constant diagonal.
A: You can understand the shape of the ellipsoid better if you look at the spectral/eigen decomposition of the precision matrix (inverse of the covariance matrix). You want to look at the eigenvalues of this inverse, not the diagonal elements.
Just a supplement to the other answers: for a multivariate Normal with dimension $k$, you can see why algebraically if you follow this. Set the density equal to some level $l$, then:
\begin{align*}
(2\pi)^{-k/2} |\Sigma|^{-1/2} \exp\left(-\frac{1}{2}(x-\mu)'\Sigma^{-1}(x-\mu) \right) &= l\\
\iff \exp\left(-\frac{1}{2}(x-\mu)'\Sigma^{-1}(x-\mu) \right) &= l'\\
\iff (x-\mu)'\Sigma^{-1}(x-\mu)  &= l''.\tag{*}
\end{align*}
(*) is the formula for an ellipsoid centered at $\mu$. The
For your first covariance matrix, the spectral decomposition of its inverse is $\Sigma^{-1} = P\Lambda P'$, where 
$$P = 
\left[\begin{array}{cc}
P_1 & P_2
\end{array}\right] =
\left[\begin{array}{cc}
.707 & -.707\\
.707 & .707
\end{array}\right]
$$
and 
$$
\Lambda = 
\left[\begin{array}{cc}
\lambda_1 & 0 \\
0 & \lambda_2
\end{array}\right] =
\left[\begin{array}{cc}
2 & 0 \\
0 & 2/3
\end{array}\right].
$$
The reason why it looks "squished" is because the diagonals of $\Lambda$ are not the same. This is because the semi-axes are $P_1/\lambda_1$ (the up and to the right vector) and $P_2/\lambda_2$ (up and to the left). Because $\lambda_1$ is bigger, that means $P_1/\lambda_1$ is a shorter vector.
What if we're used to looking at the covariance matrix, instead of its inverse? Well their spectral decompositions are pretty related. Because $\Sigma^{-1} = P\Lambda P'$ and because $P$ is orthogonal, we have
$$
\Sigma = P \Lambda^{-1}P'.
$$
Just try multiplying these two decompositions together, and you should get the identity matrix. What this tells us is that these two matrices have the same eigenvectors (and so they have the same principal axes), and the eigenvalues are reciprocals. However, I started off with the precision matrix because that's what is in the formula for the density.
More examples:
If the elements of $x$ are independent, then $\Sigma$ is diagonal, then $\Sigma^{-1}$ is diagonal, then (*) is
$$
\frac{(x_1 - \mu_1)^2}{\sigma_1^2} + \frac{(x_2 - \mu_2)^2}{\sigma_2^2} = l''\tag{**}
$$
which is still an ellipse, but it's not tilted/rotated.
If the elements of $x$ are independent and moreover they are identical, then $\sigma_1 = \sigma_2$ and (**) turns into a circle.
A: 
Consider this figure. Notice how both the circle and the dashed diagonal are inside the square. So, the circle is how the contours of the multivariate Gaussian looks when correlation is zero. The dashed diagonal is the contour of the perfectly correlated variables. The ovals (ellipses) are in between, when correlation is not equal zero or one. The length of the square sides represents the variance (standard deviation) of the variables (marginals). 
Here, I resized your picture to make the x- and y-axis scales equal, and you can see how the oval fits into a square. I think that the fact that Andrew Ng's plot was not scaled equally just added to the confusion. You can fit all kinds of ovals into the same square. You can have all kinds of contours for the same variances of variables depending on the correlation between them.

The image is from this web site, which has nothing to do with a question asked :)
