5
$\begingroup$

Empirically the quartimax-/varimax-rotation has proven useful and it was always converging in my applications. But from my readings years ago (most prominent S.Mulaik and K.Überla monographies on factor nanalysis) I have a vague remembering, that the fact of a proof of convergence has been mentioned but I've never seen the actual proof.

Perhaps this is not difficult and may be done here, or does someone have a reference?

What I'm really after is the proof for the "inverse" of the quartimax: where the rotation is defined for the minimizing (instead of the maximizing) of the criterion; I hope I can use the quartimax-convergence-proof directly or at least have a path how to proceed with such a proof on my own...

[update] Hmm, I've got some hint to articles of ten Berge(1995, 60(3)) and of Jennrich(2001, 66(2)) (both appeared in "Psychometrika" that couple of years ago), which both seem to deal with problems of the convergence of that rotation - so this seems not to be trivial (or as easy as I hoped it would possibly be)...

(Unfortunately, I shall not have access to our library before monday)

[late update]: I should have added, that I've also tried to get more references in mathoverflow and gave a bit more discussion in this question, however also without conclusive result.

Improve this question
$\endgroup$
7
  • $\begingroup$ FYI: Somebody once mentioned me Suitbert Ertel, Factor Analysis: Healing an Ailing Model; and a "varimin" criterion to obtain "complex structure" found in GPArotation R-package. Maybe you find it interesting for your problem, but I've not read the stuff. $\endgroup$
    – ttnphns
    Oct 29, 2016 at 14:30
  • $\begingroup$ @ttnphns : ahh, very nice. I'll try to get that text, thank you! $\endgroup$
    – Gottfried Helms
    Oct 29, 2016 at 14:32
  • 1
    $\begingroup$ Mulaik (2010, p. 304) said, for Quartimax algorithm, "No proof has ever been offered that such an iterative procedure will always converge, but with empirical structure matrices it always has". $\endgroup$
    – ttnphns
    Oct 29, 2016 at 14:50
  • $\begingroup$ @ttnphns : so my rememberings were roughly correct so far... Possibly such a proof were identical/had identical difficulty as the proof in the Hadamard-matrix question. So something is still there in the orthogonal (FA-) rotations, where some bright mind could involve himself... :-) Thank you very much for looking for the reference! - Ah, and if you make this an answer I could "accept" that. $\endgroup$
    – Gottfried Helms
    Oct 29, 2016 at 14:58
  • $\begingroup$ Just asking question of vanity (as from me a practicioner, not mathematician)... Why do you need a proof? If you are going to write your own factor-rotation procedure and in many many realistic simulations it then show convergence - why require a proof? $\endgroup$
    – ttnphns
    Oct 29, 2016 at 15:48

1 Answer 1

Reset to default
1
$\begingroup$

Working with othogonal eigenvector matrices M (created as random rotation matrices) a sequence of experiments suggested, that "varimin"-rotation (which is just minimizing the same criterion which "varimax" maximizes) can run into local extrema and miss the global minimum, and does this more often as the matrix-size is increased.

I used 4x4 up to 22x22 matrices M and produced 1000 and more randomly created examples of the same size, trying to rotate them to "minimal variance of the squared matrix-entries".

  • For sizes n=4k the "varimin" rotation found always the "Hadamard"-matrix versions.

  • For sizes n=4k+2 that rotation found sometimes the "Weighing-matrices" with entries from $(1,0,-1)$ (when appropriately rescaled). However, when the dimension went to 18x18 I got that ideal solutions only in roughly 2 percent of the random examples; and for 22x22 matrices that was even less frequent.
    Here I checked the improvement of the rotation-criteria for each single iteration and stopped, when the improvement became just marginal but the distance from the optimal versions was still far away.

  • For sizes n x n with n=2k+1 the rotations found sometimes versions which reminded of Hadamard- resp. Weighing- matrices, however had 4,5 or more different entries. For instance for n=13 I got in 80 percent of all examples rotated matrices with only 4 siginificantly different entries in the form $(1,a,-a,-1)$ but for n=11 that result occured only in 1 to 2 percent of examples.

Well, I considered then, that possibly such examples, which did not rotate to the "simple" structure of Hadamard or Weighing matrix having $(1,-1)$ or $(1,0,-1)$ or at least having few-categories-solutions like $(1,a,-a,-1)$ , just are incompatible with suchoptimal solutions. But for the matrix-sizes ( n=4k+2 ) for which I got both types of results, also the bad results could be transformed to an ideal solution by just (the correct) rotation.

The last observation let's me thus conjecture, that indeed "varimin" is not guaranteed to find the solution with the smallest variance in the squared matrix-entries, and the same should then be valid for the "varimax"-rotation in their intention to find the maximal variance.

Well, this experiments were done on orthogonal random-matrices, which are also eigenvector matrices for correlation-matrices. Rotation in factor-/component-analysis is usually done on loadings, which are rescaled entries of the eigenvectors - but there are also software packets which indeed rotate on the eigenvectors.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.