I have a dataset comprised of two $n$ x $d$ matrices, $A$ and $B$. Each is comprised of $n$ observations of $d$ angles. The angles are taken from two different meshes of planar triangles that are in correspondence such that columns $A_i$ and $B_i$ both represent $n$ measurements of angle $\alpha_i$. Many columns of $A$ (and $B$) are highly correlated with one another because all three angles of each triangle in the mesh from which $A$ is derived are included in $A$, i.e., for any row $m$ and some columns $i$, $j$ and $k$, $\alpha_{mi} + \alpha_{mj} + \alpha_{mk} = 180$ in $A$ (and $B$).

I wish to analyze the correlation structure of these meshes and show that some of the angles in $A$ have significant correlation with their counterparts in $B$. (I may eventually attempt to show B can be predicted from A, but that's a separate discussion.) Having computed the Pearson correlation, $\rho_i$, between $A_i$ and $B_i$ for $i = {1..d}$, and estimating a p-value, $P_i$ for each $\rho_i$ via permutation testing (repeatedly shuffling the rows of $A$ while holding $B$ fixed, and generating a null distribution of $q$ correlation values $\{\rho_i^1, ..., \rho_i^q\}$ against which $\rho_i$ can be compared), I wish to appropriately correct for multiple testing.

I have a "Wikipedia++" level of understanding of Bonferroni correction, max-T and pFDR approaches to address multiple testing, but this problem seems inherently different from the type of multiple testing problems they attempt to address (e.g., a GWAS study where there is a matrix $X$ of $n$ observations on $d$ SNPs and a (single-column) target phenotype $Y$). Here we have $d$ different null distributions generated by $d$ different pairwise correlations between $A$ and $B$.

The correlation structure within $A$ (and $B$) may well prevent us from calling these independent tests (and obviating the need for multiple correction). However, it seems far too severe to assess significance by comparing a particular correlation $\rho_i$ with the null distribution generated by keeping only the best correlation found for each permutation (a la T-max). I have seen some mention of ordering the correlations for each permutation and comparing the best actual $\rho_i$ against the distribution of $q$ best correlations observed in the permutation test, then comparing the second best to those $q$ second best correlations observed in the permutation test, and so on, and this seems plausible, but I'm not sufficiently confident to put such analysis in a paper.

Can someone kindly describe the type of permutation testing that needs to be performed to correctly assess whether some of these angle correlations are, in fact, significant? Is is acceptable to simply pool all correlations ($\rho_1, ..., \rho_d$) for all $q$ permutations and use that distribution as the null distribution against which all significance is assessed?

Guidance and references to support would be most appreciated.

(BTW, I am open to suggestions for reducing the dimensionality of the problem before proceeding with correlation analysis, but as this will not eliminate the problem, I decided not to confuse the matter with discussions of such above.)

  • $\begingroup$ As it turns out correlation is normalized covariance. Variance in turn has the same geometry as a scalene triangle $C^2=A^2+B^2-2AB\cos(\theta)$, where the correlation coefficient substitutes for $\cos(\theta)$. $\endgroup$ – Carl Mar 13 '17 at 2:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.