Maybe it is just because I'm only experimenting with 2 dimensional normal distributions, but multi-dimensional normal distributions for me seem like just multiple one dimensional normal distributions.
However, in this Wikipedia link, the illustration just at the top right corner is a two dimensional normal distribution as well, but it defines covariance ($\sum$), instead of variance ($\sigma^2)$, and I don't understand why it's needed.
I tried to reproduce the same diagram, and (seemingly) successfully implemented it just using variance. The result looks like this:
Built with this python code:
import matplotlib.pyplot as plt
import numpy as np
def gaussian(x, mean, variance):
return 1 / (variance / np.sqrt(2 * np.pi)) * np.exp(-0.5 * (x - mean)**2 / variance**2)
means = [4, 4]
variances = [1, 0.8]
placeholder = np.arange(0, 8, 0.01)
epsilon = np.random.normal(size=(300, 2))
x = np.array(means) + np.exp(0.5 * np.array(variances)) * epsilon
fig = plt.figure()
fig = fig.gca(projection='3d')
fig.plot(placeholder, gaussian(placeholder, means[0], variances[0]), zs=8, zdir='y')
fig.plot(placeholder, gaussian(placeholder, means[1], variances[1]), zs=0, zdir='x')
fig.scatter(x[:,0], x[:,1], zs = 0, zdir='z')
fig.legend()
fig.set_xlim(0, 8)
fig.set_ylim(0, 8)
fig.set_zlim(0, 4)
fig.set_xlabel('X')
fig.set_ylabel('Y')
fig.set_zlabel('Z')
plt.savefig('diagram.png')
I really need to understand this for my long journey to understand VAEs.