# Why is there covariance instead of simply variance in multidimensional normal distributions?

Maybe it is just because I'm only experimenting with 2 dimensional normal distributions, but multi-dimensional normal distributions for me seem like just multiple one dimensional normal distributions.

However, in this Wikipedia link, the illustration just at the top right corner is a two dimensional normal distribution as well, but it defines covariance ($$\sum$$), instead of variance ($$\sigma^2)$$, and I don't understand why it's needed.

I tried to reproduce the same diagram, and (seemingly) successfully implemented it just using variance. The result looks like this: Built with this python code:

import matplotlib.pyplot as plt
import numpy as np

def gaussian(x, mean, variance):
return 1 / (variance / np.sqrt(2 * np.pi)) * np.exp(-0.5 * (x - mean)**2 / variance**2)

means = [4, 4]
variances = [1, 0.8]

placeholder = np.arange(0, 8, 0.01)

epsilon = np.random.normal(size=(300, 2))
x = np.array(means) + np.exp(0.5 * np.array(variances)) * epsilon

fig = plt.figure()
fig = fig.gca(projection='3d')

fig.plot(placeholder, gaussian(placeholder, means, variances), zs=8, zdir='y')
fig.plot(placeholder, gaussian(placeholder, means, variances), zs=0, zdir='x')
fig.scatter(x[:,0], x[:,1], zs = 0, zdir='z')

fig.legend()
fig.set_xlim(0, 8)
fig.set_ylim(0, 8)
fig.set_zlim(0, 4)
fig.set_xlabel('X')
fig.set_ylabel('Y')
fig.set_zlabel('Z')

plt.savefig('diagram.png')


I really need to understand this for my long journey to understand VAEs.

• VAE = variational autoencoder? Dec 9, 2020 at 10:39
• Yes, it is variational autoencoder Dec 9, 2020 at 11:07

$$f(x,y)=A\exp(a(x-\mu_x)^2+b(y-\mu_y)^2+c(x-\mu_x)(y-\mu_y))$$
But, if you omit the covariances (i.e. assume $$0$$), the cross term disappears, i.e. $$f(x,y)=A\exp(a(x-\mu_x)^2+b(y-\mu_y)^2)$$