# The number of free parameters in factor analysis after an orthogonal rotation

Background

I'm reading some notes in multivariate data analysis, in particular factor analysis.

• A data vector $X_{p\times 1}$, with $E(X) = \mu$

• A vector $F_{m \times 1}$ of factors,

• A matrix $L_{p\times m}$ of loadings, and

• A vector $\varepsilon_{p \times 1}$ of $p$ errors with a diagonal covariance matrix $Var(\varepsilon) = \Psi$

which gives the model

$$X-\mu = LF + \varepsilon$$

The model has $mp + p$ parameters after considering $L$ and $\Psi$.

The solutions are not unique, and are only determined up to orthogonal rotations $L^* = LT$ where $T$ is an orthogonal matrix. This is used profitably to rotate the factors in a way that provides better interpretation.

Question

Then, the notes say:

after rotating with $T$, there are $\frac{m(m-1)}{2}$ fewer parameters.

I just can't figure out where that comes from. Can someone please explain?

• @amoeba No, you're right, I probably don't want it there. However, I would appreciate either a hint or a full solution. Whichever gets me closer to understanding. – RMurphy Oct 17 '16 at 23:58
• I don't have time for a full answer right now, but here is a hint: how many parameters does $T$ have? – amoeba Oct 18 '16 at 0:06
• @amoeba Hmm... I thought it was fixed and known, based on what I've read so far. I'll continue reading, and keep that hint in mind. – RMurphy Oct 18 '16 at 1:42
• Hi, I think I figured one thing out. Observe that, in order for a matrix $T$ to be orthogonal, its columns need to sum to one be mutually orthogonal. Hence, that gives a system of equations that the elements of the matrix need to satisfy. I found that $\frac{m(m-1)}{2}$ parameters are freely varying, in the sense that if you wanted to make an orthogonal matrix, you could start by picking $\frac{m(m-1)}{2}$ elements but then the rest would be determined. So I kind of get that. Is that the right direction? – RMurphy Oct 18 '16 at 18:10
• @amoeba I understand that $T$ has $m(m-1)/2$ free parameters and that L is only unique to multiplication by an orthogonal matrix. However, I am unable to understand how these two facts put together reduces the number of free parameters in $L$ by $m(m-1)/2$? – honeybadger Jul 31 '19 at 17:32

The extra constraint is usually of the form: $$L^{T} \Psi^{-1} L$$ is a diagonal matrix. By construction $$L^{T} \Psi^{-1} L$$ is a symmetric matrix, and setting the non-diagonal elements to zero is equivalent to a reduction in $$m(m-1)/2$$ degrees of freedom.