I am trying to understand how to calculate the conditional distribution of probabilistic principal component analysis. This is explained in the book "Pattern Recognition and Machine Learning" by Christopher Bishop, 2006.
This is the distribution from which the observed variables are sampled: $p(x|z) = N(x|Wz + \mu, \sigma^2 I) \tag{12.32}$ But how exactly does this matrix math work?
Lets say that our observed data is a 4x3 matrix with four samples and three different observations (i.e., features) from each sample. For example: \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \\ a_{41} & a_{42} & a_{43} \end{bmatrix}
Our W matrix (i.e., weight matrix) is a 2x3 matrix, where the two columns are principal components and the rows are the features of the observed data. For example: \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \\ b_{31} & b_{32} \end{bmatrix}
The Z matrix (i.e., latent variable matrix) is a 4x2 matrix, where the two columns are the first two principal components and the rows are the samples. For example:
\begin{bmatrix} c_{11} & c_{12} \\ c_{21} & c_{22} \\ c_{31} & c_{32} \\ c_{41} & c_{42} \end{bmatrix}
In this case σ2 is a scalar and I is the identity matrix. But let's focus on Wz. How can we multiply W by z when the number of columns in W does not equal the number of rows in Z? Could someone maybe show be a very simple example with the dimensions of matrices I've provided, explain the idea behind everything, and the terminology used for the operations?
Thank you very much.
