5
$\begingroup$

Background

I'm reading some notes in multivariate data analysis, in particular factor analysis.

  • A data vector $X_{p\times 1}$, with $E(X) = \mu$

  • A vector $F_{m \times 1} $ of factors,

  • A matrix $L_{p\times m}$ of loadings, and

  • A vector $\varepsilon_{p \times 1}$ of $p$ errors with a diagonal covariance matrix $Var(\varepsilon) = \Psi$

which gives the model

$$ X-\mu = LF + \varepsilon $$

The model has $mp + p$ parameters after considering $L$ and $\Psi$.

The solutions are not unique, and are only determined up to orthogonal rotations $L^* = LT$ where $T$ is an orthogonal matrix. This is used profitably to rotate the factors in a way that provides better interpretation.

Question

Then, the notes say:

after rotating with $T$, there are $\frac{m(m-1)}{2}$ fewer parameters.

I just can't figure out where that comes from. Can someone please explain?

Improve this question
$\endgroup$
10
  • $\begingroup$ @amoeba No, you're right, I probably don't want it there. However, I would appreciate either a hint or a full solution. Whichever gets me closer to understanding. $\endgroup$
    – RMurphy
    Oct 17, 2016 at 23:58
  • $\begingroup$ I don't have time for a full answer right now, but here is a hint: how many parameters does $T$ have? $\endgroup$
    – amoeba
    Oct 18, 2016 at 0:06
  • $\begingroup$ @amoeba Hmm... I thought it was fixed and known, based on what I've read so far. I'll continue reading, and keep that hint in mind. $\endgroup$
    – RMurphy
    Oct 18, 2016 at 1:42
  • 1
    $\begingroup$ Hi, I think I figured one thing out. Observe that, in order for a matrix $T$ to be orthogonal, its columns need to sum to one be mutually orthogonal. Hence, that gives a system of equations that the elements of the matrix need to satisfy. I found that $\frac{m(m-1)}{2}$ parameters are freely varying, in the sense that if you wanted to make an orthogonal matrix, you could start by picking $\frac{m(m-1)}{2}$ elements but then the rest would be determined. So I kind of get that. Is that the right direction? $\endgroup$
    – RMurphy
    Oct 18, 2016 at 18:10
  • 1
    $\begingroup$ @amoeba I understand that $T$ has $m(m-1)/2$ free parameters and that L is only unique to multiplication by an orthogonal matrix. However, I am unable to understand how these two facts put together reduces the number of free parameters in $L$ by $m(m-1)/2$? $\endgroup$
    – honeybadger
    Jul 31, 2019 at 17:32

1 Answer 1

Reset to default
1
$\begingroup$

I found the following references - Rotation in Factor analysis by Darton and this presentation wherein it is stated that because of the rotational indeterminacy, an extra constraint is imposed - primarily for computational convenience.

The extra constraint is usually of the form: $L^{T} \Psi^{-1} L$ is a diagonal matrix. By construction $L^{T} \Psi^{-1} L$ is a symmetric matrix, and setting the non-diagonal elements to zero is equivalent to a reduction in $m(m-1)/2$ degrees of freedom.

Update:

This answer by Amoeba explains it in more detail.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.