I have a study where six numeric variables are measured in each of two repeated measures experimental conditions (n = 200). Lets call the conditions $A$ and $B$ and the variables $A_1, A_2,..., A_6$ and $B_1, B_2,..., B_6$. Theoretically, I expect that in condition $B$ more variance in the variables should be explained by the first factor of a principal components analysis (PCA).

Typical values would be:

  • First factor of PCA on $A_1, ..., A_6$ accounts for 30% of variance
  • First factor of PCA on $B_1, ..., B_6$ accounts for 40% of variance.


  • How can I test whether this difference is statistically significant?
  • How could this be implemented in R?

4 Answers 4


Just one (maybe silly) idea. Save 1st principal component scores variable for condition A (PC1A) and 1st principal component scores variable for condition B (PC1B). The scores should be "raw", that is, their variances or sum-of-squares equal to their eigenvalues. Then use Pitman's test to compare the variances.


Did I get your answer right? - You want to test if there is statistically significant difference between the two conditions?

Perhabs vegan::adonis() is something for you? Don´t know if that´s what your looking for.

It works on the distance-matrix and compares distances within a condition are bigger then between conditions. For example in a NMDS you would see a clear separation of the two conditions.

Here is some example Code:

df <- data.frame(cond = rep(c("A", "B"), each = 100), 
 v1 <- jitter(rep(c(20, 100), each = 100)),
 v2 <- jitter(rep(c(0, 80), each = 100)),
 v3 <- jitter(rep(c(40, 5), each = 100)),
 v4 <- jitter(rep(c(42, 47), each = 100)),
 v5 <- jitter(rep(c(78, 100), each = 100)),
 v6 <- jitter(rep(c(10, 100), each = 100)))

pca <- rda(df[ ,-1], scale = TRUE)
ssc <- scores(pca, display = "sites")
ordiplot(pca, type = "n")
points(ssc[df$cond == "A", ], col = "red", pch = 16)
points(ssc[df$cond == "B", ], col = "blue", pch = 16)

nmds <- metaMDS(df[ ,-1], distance = "euclidian")
nmsc <- scores(nmds, display = "sites")
ordiplot(nmds, type = "n")
points(nmsc[df$cond == "A", ], col = "red", pch = 16)
points(nmsc[df$cond == "B", ], col = "blue", pch = 16)

# use adonis to test if there is a difference between the conditions
adonis(df[ ,-1] ~ df[ ,1], method = "euclidean")
## There is a statistically significant difference between the two conditions
  • $\begingroup$ This is rather cryptic. Could you elaborate a little? $\endgroup$
    – whuber
    Commented Sep 30, 2011 at 16:59
  • $\begingroup$ @whuber: edited a little bit. The paper from Anderson mentioned in the references of adonis is quite readable. $\endgroup$
    – EDi
    Commented Sep 30, 2011 at 18:03
  • 1
    $\begingroup$ I appreciate your effort. I'm still confused because some of the phrases seem inconsistent (the "don't" looks particularly out of place) and much of the answer has no clear connection with the question: how is "distance-matrix" related to the covariance matrix of PCA? Does "NMDS" refer to non-metric MDS and if so, what does that have to do with PCA? It seems like you're taking a completely different approach. This is fine, but it would help to explain where you're coming from and why it's applicable. $\endgroup$
    – whuber
    Commented Sep 30, 2011 at 19:07

Permutation test

To test the null hypothesis directly, use a permutation test.

Let the first PC in condition $A$ explain $a<100\%$ of variance, and the first PC in condition $B$ explain $b<100\%$ of variance. Your hypothesis is that $b>a$, so we can define $c=b-a$ as the statistic of interest, and the hypothesis is that $c>0$. The null hypothesis to reject is that $c=0$.

To perform the permutation test, take your $N=200+200$ samples from both conditions, and randomly split them into conditions $A$ and $B$. As the splitting is random, there should be no difference in explained variance after that. For each permutation, you can compute $c$, repeat this process many (say, $10000$) times, and obtain the distribution of $c$ under the null hypothesis of $c_\mathrm{true}=0$. Comparing your empirical value of $c$ with this distribution will yield a $p$-value.


To obtain the confidence interval on $c$, use bootstrapping.

In the bootstrapping approach, you would randomly select $N=200$ samples with replacement from the existing samples in $A$ and another $N=200$ from $B$. Compute $c$, and repeat it many (again, say, $10000$) times. You are going to obtain a bootstrapped distribution of the $c$ values, and its percentile intervals are going to correspond to the confidence intervals of the empirical value $c$. So you can estimate the $p$-value by looking at what part of this distribution lies above $0$.

The permutation test is a more direct (and probably less relying on any assumptions) way to test the null hypothesis, but the bootstrap has an added benefit of yielding a confidence interval on $c$.

  • $\begingroup$ The permutation test seems like a reasonable way of testing the null hypothesis. My default would probably be to try a bootstrapping approach instead where you sample with replacement. I think the benefit of a bootstrapping approach would be that you'd also get a confidence interval on the size of the difference between the two variances. My sense is that you wouldn't get that with the permutation approach. $\endgroup$ Commented Feb 6, 2015 at 23:38
  • $\begingroup$ @Jeromy, you are absolutely right about the confidence interval on the difference. In fact, I had the bootstrapping part written in my answer all this time, until I edited it out yesterday when reviewing this thread. As you mentioned it now, I have returned the bootstrapping back in right now (and also reformatted the answer a bit). Take a look. $\endgroup$
    – amoeba
    Commented Feb 6, 2015 at 23:46

This is only an outline of idea. The proportion of variance is defined as


where $\lambda_i$ are the eigenvalues of covariance matrix. Now if we use instead the eigenvalues of correlation matrix then $\lambda_1+...+\lambda_6=6$, since the sum of eigenvalues of a matrix is equal to the trace of the matrix, and for correlation matrices the trace is the sum of ones.

So if we use the correlation matrices we need to test hypotheses about the difference of two maximal eigenvalues of sample correlation matrices. It is certainly possible to find in the literature the asymptotic distribution of the maximal eigen-value of correlation matrix. So the problem then reduces to some sort of paired or unpaired t-test.

  • 1
    $\begingroup$ I suspect (based on not much information ;-) that the asymptotic distribution would be reached rather slowly for many underlying data distributions. More importantly, the correlation simply is not the covariance; PCA on the two can differ radically, especially when the variances of the variables differ. $\endgroup$
    – whuber
    Commented Sep 30, 2011 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.