16
$\begingroup$

Factor analysis has several rotation methods, such as varimax, quartimax, equamax, promax, oblimin, etc. I am unable to find any information that relates their names to their actual mathematical or statistical doings. Why is it called "equa-max" or "quarti-max"? In what way are the axes or matrices rotated so they have such name?

Unfortunately, most of them were invented in 1950s-1970s, so I cannot contact their authors.

$\endgroup$
2
  • 3
    $\begingroup$ I edited your question and its title so it is better formed. Feel free to edit it or revert the changes if you don't agree with them. The question is quite interesting so in my opinion it is important for it to be well-formed. $\endgroup$
    – Tim
    Commented Dec 5, 2015 at 20:27
  • 1
    $\begingroup$ @Tim, I edited further, and in particular added the recently created [factor-rotation] tag which you perhaps have not encountered yet. $\endgroup$
    – amoeba
    Commented Dec 5, 2015 at 21:41

2 Answers 2

25
$\begingroup$

This answer succeeds this general question on rotations in factor analysis (please read it) and briefly describes a number of specific methods.

Rotations are performed iteratively and on every pair of factors (columns of the loading matrix). This is needed because the task to optimize (maximize or minimize) the objective criterion simultaneously for all the factors would be mathematically difficult. However, in the end the final rotation matrix $\bf Q$ is assembled so that you can reproduce the rotation yourself with it, multiplying the extracted loadings $\bf A$ by it, $\bf AQ=S$, getting the rotated factor structure matrix $\bf S$. The objective criterion is some property of the elements (loadings) of resultant matrix $\bf S$.

Quartimax orthogonal rotation seeks to maximize the sum of all loadings raised to power 4 in $\bf S$. Hence its name ("quarti", four). It was shown that reaching this mathematical objective corresponds enough to satisfying the 3rd Thurstone's criterion of "simple structure" which sounds as: for every pair of factors there is several (ideally >= m) variables with loadings near zero for any one of the two and far from zero for the other factor. In other words, there will be many large and many small loadings; and points on the loading plot drawn for a pair of rotated factors would, ideally, lie close to one of the two axes. Quartimax thus minimizes the number of factors needed to explain a variable: it "simplifies" the rows of the loading matrix. But quartimax often produces the so called "general factor" (which most of the time is not desirable in FA of variables; it is more desirable, I believe, in the so called Q-mode FA of respondents).

Varimax orthogonal rotation tries to maximize variance of the squared loadings in each factor in $\bf S$. Hence its name (variance). As the result, each factor has only few variables with large loadings by the factor. Varimax directly "simplifies" columns of the loading matrix and by that it greatly facilitates the interpretability of factors. On the loading plot, points are spread wide along a factor axis and tend to polarize themselves into near-zero and far-from-zero. This property seems to satisfy a mixture of Thurstones's simple structure points to an extent. Varimax, however, is not safe from producing points lying far away from the axes, i.e. "complex" variables loaded high by more than one factor. Whether this is bad or ok depends of the field of the study. Varimax performs well mostly in combination with the so called Kaiser's normalization (equalizing communalities temporarily while rotating), it is advised to always use it with varimax (and recommended to use it with any other method, too). It is the most popular orthogonal rotation method, especially in psychometry and social sciences.

Equamax (rarely, Equimax) orthogonal rotation can be seen as a method sharpening some properties of varimax. It was invented in attempts to further improve it. Equalization refers to a special weighting which Saunders (1962) introduced into a working formula of the algorithm. Equamax self-adjusts for the number of the being rotated factors. It tends to distribute variables (highly loaded) more uniformly between factors than varimax does and thus further is less prone to giving "general" factors. On the other hand, equamax wasn't conceived to give up the quartimax's aim to simplify rows; equamax is rather a combination of varimax and quartimax than their in-between. However, equamax is claimed to be considerably less "reliable" or "stable" than varimax or quartimax: for some data it can give disastrously bad solutions while for other data it gives perfectly interpretable factors with simple structure. One more method, similar to equamax and even more ventured in quest of simple structure is called parsimax ("maximizing parsimony") (See Mulaik, 2010, for discussion).

I am sorry for stopping now and not reviewing the oblique methods - oblimin ("oblique" with "minimizing" a criterion) and promax (unrestricted procrustes rotation after varimax). The oblique methods would require probably longer paragraphs to describe them, but I didn't plan any long answer today. Both methods are mentioned in Footnote 5 of this answer. I may refer you to Mulaik, Foundations of factor analysis (2010); classic old Harman's book Modern factor analysis (1976); and whatever pops out in the internet when you search.

See also The difference between varimax and oblimin rotations in factor analysis; What does “varimax” mean in SPSS factor analysis?

Later addendum, with the history and the formulae, for meticulous

Quartimax

In the 1950s, several factor analysis experts tried to embody Thurstone’s qualitative features of a “simple structure” (See footnote 1 here) into strict, quantitative criteria:

  • Ferguson reasoned that the most parsimonious disposition of points (loadings) in the space of factors (axes) will be when, for most pairs of factors, each of the two axes pierces its own clot of points, thus maximizing its own coordinates and minimizing the coordinates onto the perpendicular axis. So he suggested to minimize the products of loadings for each variable in pairs of factors (i,j), summed across all variables: $\sum^p\sum_{i,j;i<j}(a_i a_j)^2$ ($a$ is a loading, an element of a p variables x m factors loading matrix $\bf A$, in this case we mean, the final loadings - after a rotation).
  • Carroll also thought of pairs of factors (i,j) and wanted to minimize $\sum_{i,j;i<j}\sum^p(a_i^2 a_j^2)$. The idea was that for each pair of factors, the loadings should mostly be unequal-sized or both small, ideally a zero one against a nonzero or zero one.
  • Neuhaus and Wrigley wanted to maximize the variance of the squared values of loadings in the whole $\bf A$, in order the loadings to split themselves into big ones and near-zero ones.
  • Kaiser also chose variance, but variance of the squared loadings in rows of $\bf A$; and wanted to maximize the sum of these variances across the rows.
  • Saunders offered to maximize the kurtosis in the doubled distribution of the loadings (i.e., every loading from $\bf A$ is taken twice - with positive and with negative sign, since the sign of a loading is basically arbitrary). High kurtosis in this symmetric around zero distribution implies maximization of the share (contribution) of extreme (big) loadings as well of near-zero loadings, at the expense of the moderate-size loadings.

It then occurred (and it can be shown mathematically) that, in the milieu of orthogonal rotation, the optimization of all these five criteria is in fact equivalent from the “argmax” point of view, and they all can boil down to the maximization of

$Q= \sum^p\sum^m a^4$,

the overall sum of the 4-th power of loadings. The criterion therefore was called the quartimax. To repeat what was said in the beginning of the answer, quartimax minimizes the number of factors needed to explain a variable: it "simplifies" the rows of the loading matrix. But quartimax not rarely produces the so called "general factor".

Varimax

Having observed that quartimax simplifies well rows (variables) but is prone to “general factor”, Kaiser suggested to simplify $\bf A$’s columns (factors) instead. It was put above, that Kaiser’s idea for quartimax was to maximize the summed variance of squared loadings in rows of $\bf A$. Now he transposed the proposal and suggested to maximize the summed variance of squared loadings in columns of $\bf A$. That is, to maximize $\sum^m[\frac{1}{p} \sum^p (a^2)^2 - \frac{1}{p^2} (\sum^p a^2)^2]$ (the bracketed part is the formula of the variance of p squared values $a$), or, if multiplied by $p^2$, for convenience:

$V = \sum^m[p \sum^p (a^2)^2 - (\sum^p a^2)^2] = p \sum^m\sum^p (a^4) - \sum^m(\sum^p a^2)^2 = pQ - W$

where $V$ is the varimax criterion, $Q$ is the quartimax criterion, and $W$ is the sum of squared variances of the factors (after the rotation) [a factor's variance is the sum of its squared loadings].

[I’ve remarked that Kaiser obtained varimax by simply transposing the quartimax’s problem - to simplify columns in place of rows, - and you may switch places of m and p in the formula for $V$, to get the symmetric corresponding expression, $mQ – W^*$, for quartimax. Since we are rotating columns, not rows, of the loading matrix, the quartimax’s term $W^*$, the sum of squared communalities of the variables, does not change with rotation and therefore can be dropped from the objective statement; after which can also drop multiplier m - and stay with sole $Q$, what quartimax is. While in case of varimax, term $W$ changes with rotations and thus stays an important part of the formula, to be optimized along with it.]

Kaiser normalization. Kaiser felt dissatisfied with that variables with large communalities dictate the rotation by $V$ criterion much more than variables with small communalities. So he introduced normalizing all communalities to unit before launching the procedure maximizing $V$ (and, of course, de-normalizing back after the performed rotation - communalities don’t change in an orthogonal rotation). Per tradition, Kaiser normalization is often recommended to do – mainly with varimax, but sometimes along with quartimax and other rotation methods too, because, logically, it is not tied with varimax solely. Whether the trick is really beneficial, is an unsettled issue. Some software do it by default, some – by default only for varimax, still some – don’t set it to be a default option. (In the end of this answer, I have a remark on the normalization.)

So was varimax, who maximizes variances of squared loadings in columns of $\bf A$ and therefore simplifies the factors – in exact opposition to quartimax, who did that in rows of $\bf A$, simplifying the variables. Kaiser demonstrated that, if the population factor structure is relatively sharp (i.e., variables tend to cluster together around different factors), varimax is more robust (stable) than quartimax to removal of some variables from the rotation operation.

Equamax and Parsimax

Saunders decided to play up the fact that quartimax and varimax are actually one formula, $pQ - cW$, where $c=0$ (and then p traditionally is dropped) for quartimax and $c=1$ for varimax. He experimented with factor analytic data in the search of a greater value for coefficient $c$ in order to accentuate the varimaxian, non-quartimaxian side of the criterion. He found that $c=m/2$ often produces factors that are more interpretable than after varimax or quartimax rotations. He called $pQ – \frac{m}{2}W$ equamax. The rationale to make $c$ dependent on m was that as the number of factors grows while p does not, the a priori expected proportion of variables to be loaded by any one factor diminishes; and to compensate it, we should raise $c$.

In a similar pursuit of further “bettering” the generic criterion, Crawford arrived at yet another coefficient value, $c = p(m−1)/(p+m−2)$, depending both on m and p. This version of the criterion was named parsimax.

It is possible further to set $c=p$, yielding criterion facpars, “factor parsimony”, which, as I’m aware, is very seldom used.

(I think) It is still an open question if equamax or parsimax are really better than varimax, and if yes, then in what situations. Their dependence on the parameters m (and p) makes them self-tuning (for advocates) or capricious (for critics). Well, from purely math or general data p.o.v., raising $c$ means simply pushing factors in the direction of more equal final variances, - and not at all making the criterion “more varimax than varimax” or “balanced between varimax and quartimax” w.r.t. their objective goals, for both varimax and quartimax optimize well to the limit what they were meant to optimize.

The considered generic criterion of the form $pQ - cW$ (where Q is quartimax, $\sum^p\sum^m a^4$, and W is the sum of squared factor variances, $\sum^m(\sum^p a^2)^2$, is known as orthomax. Quartimax, varimax, equamax, parsimax, and facpars are its particular versions. In general, coefficient $c$ can take on any value. When close to +infinity, it produces factors of completely equal variances (so use that, if your aim is such). When close to -infinity, you get loadings equal to what you get if you rotate your loading matrix into its principal components by means of PCA (without centering the columns). So, value of $c$ is the parameter stretching the dimension “great general factor vs all factors equal strength”.

In their important paper of 1970, Crawford & Ferguson extend the varying $c$ criterion over to the case of nonorthogonal factor rotations (calling that more general coefficient kappa).

Literature

  • Harman, H.H. Modern factor analysis. 1976.
  • Mulaik, S.A. Foundations of factor analysis. 2010.
  • Clarkson, D.B. Quartic rotation criteria and algorithms // Psychometrica, 1988, 53, 2, p. 251-259.
  • Crawford, C.B., Ferguson, G.A. A general rotation criterion and its use in orthogonal rotation // Psychometrica, 1970, 35, 3, p. 321-332.

Comparing main characteristics of the criteria

I’ve been generating p variables x m factors loading matrices as values from uniform distribution (so yes, that was not a sharp, clean factor structure), 50 matrices for each combination of p and m/p proportion, and rotating each loading matrix by quartimax (Q), varimax (V), equamax (E), parsimax (P), and facpars (F), all methods accompanied by Kaiser normalization. Quartimax (Q0) and varimax (V0) were also tried without Kaiser normalization. Comparisons between the criteria on three characteristics of the rotated matrix are displayed below (for each matrix generated, the 7 values of the post-rotational characteristic were rescaled into the 0-1 range; then means across the 50 simulations and 95% CI are plotted).

Fig.1. Comparing the sum of variances of squared loadings in rows (maximizing this is the quartimax’s prerogative): enter image description here

Comment: Superiority of quartimax over the other criteria tend to grow as p increases or as m/p increases. Varimax most of the time is second best. Equamax and parsimax are quite similar.

Fig.2. Comparing the sum of variances of squared loadings in columns (maximizing this is the varimax’s prerogative): enter image description here

Comment: Superiority of varimax over the other criteria tend to grow as p increases or as m/p increases. Quartimax’s tendency is opposite: as the parameters increase it loses ground. In the bottom-right part, quartimax is the worst, that is, with large-scale factor analysis it fails to mimic “varimaxian” job. Equamax and parsimax are quite similar.

Fig.3. Comparing inequality of factor variances (this is driven by coefficient $c$); the variance used as the measure of “inequality”: enter image description here

Comment: Yes, with growing $c$, that is, in the line Q V E P F, the inequality of factor variances falls. Q is the leader of the inequality, which tells of its propensity for “general factor”, and at that its gap with the other criteria enlarges as p grows or m/p grows.

Comparing inequality of factor variances (this is driven by coefficient $c$); proportion “sum of absolute loadings of the strongest factor / average of such sums across the rest m-1 factors” was used as the measure of “inequality”:

This is another and more direct test for the presence of “general factor”. The configuration of results was almost the same as on the previous picture Fig.3, so I’m not showing a picture.

Disclaimer. These tries, on which the above pics are based, were done on loading matrices with random nonsharp factor structures, i.e. there were no specially preset clear clusters of variables or other specific structure among the loadings.

Kaiser normalization. From the above Fig.1-2 one can learn that versions of quartimax and varimax without the normalization perform the two tasks (the maximizations) markedly better than when accompanied by the normalization. At the same time, absence of the normalization is a little bit more prone to “general factor” (Fig.3).

The question whether Kaiser normalization should be used (and when), seems still open to me. Perhaps one should try both, with and without the normalization, and see where the applied factor interpretation was more satisfying. When we don’t know what to choose based on math grounds, it’s time we resort to “philosophical” consideration, what are set contrasted, as usual. I could imagine of two positions:

  • Contra normalization. A variable with small communality (high uniqueness) is not much helpful with any rotation. It contains only traces of the totality of the m factors, so lacks a chance to get a large loading of any of them. But we are interpreting factors mostly by large loadings, and the smaller is the loading the harder is to sight the essence of the factor in the variable. It would be justified even to exclude a variable with small communality from the rotation. Kaiser normalization is what is counter-directed to such motive/motif.
  • Pro normalization. Communality (non-uniqueness) of a variable is the amount of its inclination to the space of m factors from the outside (i.e., it is the magnitude of its projection into that space). Rotation of axes inside that space is not related with that inclination. The rotation – solving the question which of the m factors will and which will not load the variable – concerns equally a variable with any size of communality, because the initial suspense of the said “internal” decision is sharp to the same degree to all variables with their “external” inclination. So, as long as we are choosing to speak of the variables and not their projections inside, there’s no reason to spread them weights depending on their inclinations, in the act of rotation. And, to manage to discern the essence of a factor in the variable under any size of the loading – is a desideratum (and theoretically a must) for an interpreter of factors.

Orthogonal analytic rotations (Orthomax) algorithm pseudocode

Shorthand notation:
*    matrix multiplication (or simple multiplication, for scalars)
&*   elementwise (Hadamard) multiplication
^    exponentiation of elements
sqrt(M)    square roots of elements in matrix M
rsum(M)    row sums of elements in matrix M
csum(M)    column sums of elements in matrix M
rssq(M)    row sums of squares in matrix M, = rsum(M^2)
cssq(M)    column sums of squares in matrix M, = csum(M^2)
msum(M)    sum of elements in matrix M
make(nr,nc,val)   create nr x nc matrix populated with value val

A is p x m loading matrix with m orthogonal factors, p variables

If Kaiser normalization is requested:
    h = sqrt(rssq(A)). /*sqrt(communalities), column vector
    A = A/(h*make(1,m,1)). /*Bring all communalities to unit

R is the orthogonal rotation matrix to accrue:
Initialize it as m x m identity matrix

Compute the initial value of the criterion Crit;
the coefficient c is: 0 for Quartimax, 1 for Varimax, m/2 for Equamax,
p(m-1)/(p+m-2) for Parsimax, p for Facpars; or you may choose arbitrary c:
Q = msum(A^4)
If “Quartimax”
    Crit = Q
Else
    W = rssq(cssq(A))
    Crit = p*Q – c*W

Begin iterations

    For each pair of factors (columns of A) i, j (i<j) do:
        ai = A(:,i) /*Copy out the
        aj = A(:,j) /*two factors

        u = ai^2 – aj^2
        v = 2 * ai &* aj
        @d = 2 * csum(u &* v)
        @c = csum(u^2 – v^2)
        @a = csum(u)
        @b = csum(v)

        Compute the angle Phi of rotation of the two factors in their space
        (coefficient c as defined above):
        num = @d – c * 2*@a*@b/p
        den = @c – c * (@a^2 - @b^2)/p
        Phi4 = artan(num/den) /*4Phi (in radians)
        If den>0 /*4Phi is in the 1st or the 4th quadrant
            Phi = Phi4/4
        Else if num>0 /*4Phi is in the 2nd quadrant (pi is the pi value)
            Phi = (pi + Phi4)/4
        Else /*4Phi is in the 3rd quadrant
            Phi = (Phi4 – pi)/4

        Perform the rotation of the pair (rotate if Phi is not negligible):
        @sin = sin(Phi)
        @cos = cos(Phi)
        r_ij = {@cos,-@sin;@sin,@cos} /*The 2 x 2 rotation matrix
        A(:,{i,j}) = {ai,aj} * r_ij /*Rotate factors (columns) i and j in A
        R(:,{i,j}) = R(:,{i,j}) * r_ij /*Update also the columns of the being accrued R

        Go to consider next pair of factors i, j, again copying them out, etc.

    When all pairs are through, compute the criterion:
    Crit = … (see as defined above)

End iterations if Crit has stopped growing any much (say, increase not greater than
0.0001 versus the previous iteration), or the stock of iterations (say, 50) exhausted.

If Kaiser normalization was requested:
     A = A &* (h*make(1,m,1)) /*De-normalize

Ready. A has been rotated. A(input)*R = A(output)

Optional post-actions, for convenience:
1) Reorder factors by decreasing their variances (i.e., their cssq(A)).
2) Switch sign of the loadings so that positive loadings prevail in each factor.

Quartimax and Varimax are always positive values; others can be negative.
All the criteria grow on iterations.
$\endgroup$
1
$\begingroup$

Rotation methods optimise heuristic fuctions with the aim of "simplifying" factor loadings. Simplicity can be defined in many different ways. The most commonly used ones come from Thurnstone [2]: sparsity, column simplicity and parsimony, row-simplicity (or complexity). Most rotation criteria address one or the other of both, their names are not really important.

Single criteria are included in families of criteria: the most comprehensive one is the Crawford-Ferguson one, which is equivalent to the Orthomax family for orthogonal rotations. These families provide a weighing of both simplicity requirements controlled by different parameters. By changing these, almost all known rotation criteria can be obtained. An excellent and accessible overview of rotation methods is the Browne paper.

[1] M. Browne, An overview of analytic rotation in exploratory factor analysis, Multivariate Behavioral Research 36 (2001), pp. 111–150.

[2] L. Thurstone, Multiple-factor analysis, The University of Chicago Press, 1947

$\endgroup$
1
  • 1
    $\begingroup$ Thank you for your time that you gave to answer that question of mine dear Marco. However, the central point of my question was about "what the rotation names are meaning in relation to their mathematical or statistical manipulations they are doing in the data". Why their inventors or other people gave the name "Varimax" to that specific method of rotation as well to the other types of rotation methods ? What it was its meaning; @ttnphns answered that question quite well. If you think you can add more information on that topic feel free to do it! :) Thank you again for your time. $\endgroup$ Commented Sep 26, 2018 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.