# How can a covariance matrix for a normal distribution not be quadratic?

Currently Im reading this paper and in section 3.3., I came across the definition of a multi-dimensional standard normal distribution:

\begin{align} q(\pmb{\epsilon}) = \mathcal{N}(\textbf{0}, \textbf{I}_{Q \times K}). \end{align}

What does $$\textbf{I}_{Q \times K}$$ suppose to mean? I thought that covariance matrices must always be quadratic! How can one define the unit matrix of dimensions $$Q \times K$$ to be covariance function for a standard normal distribution?

Perhaps I missunderstood something here, but does anyone have a clue by chance what the meaning of this is?

Thanks

• Agree, it is a confusing choice of notation.
– Tim
Jun 9, 2020 at 10:55
• Any intuition about what is meant? Jun 9, 2020 at 10:59

It's unfortunate notation, but my understanding is that they mean an identity matrix of size $$N = Q \cdot K$$, ie Q times K. This is needed since $$\mathbf{W}_1$$ is a $$Q \times K$$ matrix so the noise/error term $$\mathbf{\epsilon_1}$$ in that equation must be of size $$Q \cdot K$$ to match LHS in eq (14).
I think it's more clear to look first at the bias term: here $$\mathbf{\epsilon}$$ is of dimension $$K$$ because the bias term $$\mathbf{b}$$ is of dimension $$K$$. They do the same for the matrix equations, but now think of it as a flattened out $$Q \times K$$ matrix into a $$Q \cdot K$$ vector; in order to add a noise term $$\epsilon_1$$ to that (flattened) vector it has to come from a normal distribution of dimension $$K \cdot Q$$, which you get for an identity matrix of size $$(Q \cdot K) \times (Q \cdot K)$$.
• thanks for looking into it! So you suggest the identity matrix to be of dimension $Q*K \times Q*K$, I don't see yet how this results into matrix $W_1$ being dimension $Q \times K$ in eq. (14). I'd be grateful if you could elaborate on this Jun 9, 2020 at 12:56