Tag Info

Factor analysis is a dimensionality reduction latent variable technique which replaces inter-correlating variables with a smaller number of continuous latent variables called factors. The factors are believed to be responsible for the inter-correlations. [For confirmatory factor analysis, please use the tag 'confirmatory-factor'. Also, the term "factor" of factor analysis should not be confused with "factor" as categorical predictor of a regression/ANOVA.]

A latent factor (matrix) model has the following form:

$${\bf Y} = {\bf \mathbb{1}\mu}^\top + {\bf ZA}^\top + {\bf E\psi}^{1/2}$$

Where:

• $${\bf Y} \in \mathbb{R}^{n \times p}$$ whose rows are random vectors in $$\mathbb{R}^p$$
• $${\bf \mu} \in \mathbb{R}^p$$ is a column vector of means of a generic row of $${\bf Y}$$
• $${\bf Z}$$ is a matrix whose rows are unobserved latent factors $${\bf z}_i$$
• $${\bf A} \in \mathbb{R}^{p \times q}$$ is a matrix whose columns form linear combinations of the factors, $${\bf z}$$. These are the factor loadings
• $${\bf E}$$ is a normal matrix with 0 mean and Identity covariance
• $${\bf \psi} =\mathrm{diag}(\psi_1, ... ,\psi_p)$$. This ensures that all dependencies among the rows of $${\bf Y}$$ are dependent on $${\bf Z}$$.

Note that $$E[{\bf Y}] = {\bf \mathbb{1}\mu}^\top + {\bf ZA}^\top$$ and $$Var({\bf Y}) = \psi \otimes I$$. This model is only useful if $$q < p$$, otherwise $${\bf ZA}^\top$$ is full rank and this model is the same as an unrestricted and arbitrary model, $$E[{\bf Y}] = {\bf M}$$ for any $${\bf M}$$.