12
$\begingroup$

I have computed a correlation matrix of a data set which contains 455 data points, each data point containing 14 characteristics. So the dimension of the correlation matrix is 14 x 14.

I was wondering whether there is a threshold for the value of the correlation coefficient which points out that there is a significant correlation between two of those characteristics.

I have value ranging from -0.2 to 0.85, and I was thinking that the important ones are those which are above 0.7.

  • Is there a general value for the correlation coefficient which should be considered for the threshold or is just context dependent to the data type which I am investigating?
$\endgroup$
2
  • 1
    $\begingroup$ have you checked stats.stackexchange.com/questions/5750/… ? $\endgroup$
    – user603
    Aug 3, 2011 at 16:07
  • $\begingroup$ @user603 Good catch: it's practically the same question. The innovation here is to ask whether tests for significant correlation might depend on the "data type" (read: data distribution). Let's hope that the replies focus on this aspect instead of going over old ground. $\endgroup$
    – whuber
    Aug 3, 2011 at 16:44

3 Answers 3

10
$\begingroup$

Significance tests for correlations

There are tests of statistical significance that can be applied to individual correlations, which indicate the probability of obtaining a correlation as large or larger than the the sample correlation assuming the null hypothesis is true.

The key point is that what constitutes a statistically significant correlation coefficient depends on:

  • Sample size: bigger sample sizes will lead to smaller thresholds
  • alpha: often set to .05, smaller alphas will lead to higher thresholds for statistical significance
  • one-tailed / two-tailed test: I'm guessing that you would be using two-tailed so this probably doesn't matter
  • type of correlation coefficient: I'm guessing you are using Pearson's
  • distributional assumptions of x and y

In common circumstances, where alpha is .05, using two-tailed test, with Pearson's correlation, and where normality is at least an adequate approximation, the main factor influencing the cut-off is sample size.

Threshold of importance

Another way of interpreting your question is to consider that you are interested not in whether a correlation is statistically significant, but rather whether it is practically important.

Some researchers have offered rules of thumb for interpreting the meaning of correlation coefficients, but these rules of thumb are domain specific.

Multiple significance testing

However, because you are interested in flagging significant correlations in a matrix, this changes the inferential context. You have $k(k-1)/2$ correlations where $k$ is the number of variables (i.e., $14(13)/2=91$. If the null hypothesis were true for all correlations in the matrix, then the more significance tests you run, then the more likely you are of making a Type I error. E.g., in your case you would on average make $91 * .05 = 4.55$ Type I errors if the null hypothesis were true for all correlations.

As @user603 has pointed out, these issues were well discussed in this earlier question.

In general, I find it useful when interpreting a correlation matrix to focus on higher level structure. This can be done in an informal way by looking at general patterns in the correlation matrix. This can be done more formally by using techniques like PCA and factor analysis. Such approaches avoid many of the issues associated with multiple significance testing.

$\endgroup$
1
$\begingroup$

One option would be simulation or permutation testing. If you know the distribution that your data comes from you could simulate from that distribution, but with all the observations independent. If you don't know the distribution then you can permute each of your variables independly of each other and that will give you the same general marginal distribution of each variable, but with any correlation removed.

Do either of the above (keeping the sample size and matrix dimensions the same) a whole bunch of times (10,000 or so) and look at the maximum absolute correlation, or another high quantile that may be of interest. This will give you the distribution from the null hypothesis that you can then compare the maximum of your actual observed correlations to (and the other high quantiles of interest).

$\endgroup$
0
$\begingroup$

You can show that standard error in Pearson correlation of two stochastically independent vectors sampled from the normal distribution is $n^{-2}$, where $n$ is the length of the vector. So statistically significant correlation of two vectors would have $corr >> n^{-2}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.