8
$\begingroup$

Context:

I have a study where six numeric variables are measured in each of two repeated measures experimental conditions (n = 200). Lets call the conditions $A$ and $B$ and the variables $A_1, A_2,..., A_6$ and $B_1, B_2,..., B_6$. Theoretically, I expect that in condition $B$ more variance in the variables should be explained by the first factor of a principal components analysis (PCA).

Typical values would be:

  • First factor of PCA on $A_1, ..., A_6$ accounts for 30% of variance
  • First factor of PCA on $B_1, ..., B_6$ accounts for 40% of variance.

Questions:

  • How can I test whether this difference is statistically significant?
  • How could this be implemented in R?
$\endgroup$
1
$\begingroup$

Just one (maybe silly) idea. Save 1st principal component scores variable for condition A (PC1A) and 1st principal component scores variable for condition B (PC1B). The scores should be "raw", that is, their variances or sum-of-squares equal to their eigenvalues. Then use Pitman's test to compare the variances.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Did I get your answer right? - You want to test if there is statistically significant difference between the two conditions?

Perhabs vegan::adonis() is something for you? Don´t know if that´s what your looking for.

It works on the distance-matrix and compares distances within a condition are bigger then between conditions. For example in a NMDS you would see a clear separation of the two conditions.

Here is some example Code:

df <- data.frame(cond = rep(c("A", "B"), each = 100), 
 v1 <- jitter(rep(c(20, 100), each = 100)),
 v2 <- jitter(rep(c(0, 80), each = 100)),
 v3 <- jitter(rep(c(40, 5), each = 100)),
 v4 <- jitter(rep(c(42, 47), each = 100)),
 v5 <- jitter(rep(c(78, 100), each = 100)),
 v6 <- jitter(rep(c(10, 100), each = 100)))

# PCA
require(vegan)
pca <- rda(df[ ,-1], scale = TRUE)
ssc <- scores(pca, display = "sites")
ordiplot(pca, type = "n")
points(ssc[df$cond == "A", ], col = "red", pch = 16)
points(ssc[df$cond == "B", ], col = "blue", pch = 16)

# NMDS
nmds <- metaMDS(df[ ,-1], distance = "euclidian")
nmsc <- scores(nmds, display = "sites")
ordiplot(nmds, type = "n")
points(nmsc[df$cond == "A", ], col = "red", pch = 16)
points(nmsc[df$cond == "B", ], col = "blue", pch = 16)

# use adonis to test if there is a difference between the conditions
adonis(df[ ,-1] ~ df[ ,1], method = "euclidean")
## There is a statistically significant difference between the two conditions
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This is rather cryptic. Could you elaborate a little? $\endgroup$ – whuber Sep 30 '11 at 16:59
  • $\begingroup$ @whuber: edited a little bit. The paper from Anderson mentioned in the references of adonis is quite readable. $\endgroup$ – EDi Sep 30 '11 at 18:03
  • 1
    $\begingroup$ I appreciate your effort. I'm still confused because some of the phrases seem inconsistent (the "don't" looks particularly out of place) and much of the answer has no clear connection with the question: how is "distance-matrix" related to the covariance matrix of PCA? Does "NMDS" refer to non-metric MDS and if so, what does that have to do with PCA? It seems like you're taking a completely different approach. This is fine, but it would help to explain where you're coming from and why it's applicable. $\endgroup$ – whuber Sep 30 '11 at 19:07
1
$\begingroup$

Permutation test

To test the null hypothesis directly, use a permutation test.

Let the first PC in condition $A$ explain $a<100\%$ of variance, and the first PC in condition $B$ explain $b<100\%$ of variance. Your hypothesis is that $b>a$, so we can define $c=b-a$ as the statistic of interest, and the hypothesis is that $c>0$. The null hypothesis to reject is that $c=0$.

To perform the permutation test, take your $N=200+200$ samples from both conditions, and randomly split them into conditions $A$ and $B$. As the splitting is random, there should be no difference in explained variance after that. For each permutation, you can compute $c$, repeat this process many (say, $10000$) times, and obtain the distribution of $c$ under the null hypothesis of $c_\mathrm{true}=0$. Comparing your empirical value of $c$ with this distribution will yield a $p$-value.

Bootstrapping

To obtain the confidence interval on $c$, use bootstrapping.

In the bootstrapping approach, you would randomly select $N=200$ samples with replacement from the existing samples in $A$ and another $N=200$ from $B$. Compute $c$, and repeat it many (again, say, $10000$) times. You are going to obtain a bootstrapped distribution of the $c$ values, and its percentile intervals are going to correspond to the confidence intervals of the empirical value $c$. So you can estimate the $p$-value by looking at what part of this distribution lies above $0$.

The permutation test is a more direct (and probably less relying on any assumptions) way to test the null hypothesis, but the bootstrap has an added benefit of yielding a confidence interval on $c$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ The permutation test seems like a reasonable way of testing the null hypothesis. My default would probably be to try a bootstrapping approach instead where you sample with replacement. I think the benefit of a bootstrapping approach would be that you'd also get a confidence interval on the size of the difference between the two variances. My sense is that you wouldn't get that with the permutation approach. $\endgroup$ – Jeromy Anglim Feb 6 '15 at 23:38
  • $\begingroup$ @Jeromy, you are absolutely right about the confidence interval on the difference. In fact, I had the bootstrapping part written in my answer all this time, until I edited it out yesterday when reviewing this thread. As you mentioned it now, I have returned the bootstrapping back in right now (and also reformatted the answer a bit). Take a look. $\endgroup$ – amoeba Feb 6 '15 at 23:46
0
$\begingroup$

This is only an outline of idea. The proportion of variance is defined as

$$\frac{\lambda_1}{\lambda_1+...+\lambda_6},$$

where $\lambda_i$ are the eigenvalues of covariance matrix. Now if we use instead the eigenvalues of correlation matrix then $\lambda_1+...+\lambda_6=6$, since the sum of eigenvalues of a matrix is equal to the trace of the matrix, and for correlation matrices the trace is the sum of ones.

So if we use the correlation matrices we need to test hypotheses about the difference of two maximal eigenvalues of sample correlation matrices. It is certainly possible to find in the literature the asymptotic distribution of the maximal eigen-value of correlation matrix. So the problem then reduces to some sort of paired or unpaired t-test.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I suspect (based on not much information ;-) that the asymptotic distribution would be reached rather slowly for many underlying data distributions. More importantly, the correlation simply is not the covariance; PCA on the two can differ radically, especially when the variances of the variables differ. $\endgroup$ – whuber Sep 30 '11 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.