Tests of significance, correlation vs chi-squared [closed]

Question

I am comparing two ordinal variables along a Likert scale. From previous help received on this forum, I have decided to use the Spearman Rho for my correlation statistic. My next question is how to test significance. I have used a Chi-squared, but there is also a significance test for the actual Spearman Rho statistic. I'm not quite clear on what the two should be telling me. I know the Chi-squared should tell me whether there is a significant relationship at all, whereas the Spearman Rho p-value will tell me if the correlation coefficient is significant, so please don't give me answers along that line. Is it possible to have one be significant but not the other?

Answer 1

No two different tests can reasonably be expected to yield the same probability of significance, that is implicit to their being genuinely different. The Pearson Chi-squared test implies the four assumptions of

(1) A simple random sample, that is, the sample data is a random sampling 
 from a fixed distribution or population where every collection of members 
 of the population of the given sample size has an equal probability of
 selection. Variants of the test have been developed for complex samples

(2) A sufficiently large sample size. If a chi squared test is conducted on
 a sample with a smaller size, then the chi squared test will yield an 
 inaccurate inference, e.g., committing a Type II error.

(3) Sufficiently large expected cell counts. Some require 5 or more, and
 others require 10 or more. A common rule is 5 or more in all cells of a
 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells
 with zero expected count. When this assumption is not met, Yates's 
 correction is applied.

(4) Independence.  The observations are always assumed to be independent of 
 each other. This means chi-squared cannot be used to test correlated data
 (like matched pairs or panel data). 

Since a certain frequency of data in one category correlates by subtraction to the other categories for the question at hand, condition (4) above, independence, is not met exactly leading in the binary response case to the Cochran's Q test. For an ordinal response, the Friedman test and the Durbin test are considerations.

By way of contrast, the Spearman's rho significance tests the strength of monotonicity of ranked data against no monotonicity and in so doing it adjusts for monotonic nonlinearity in the data. It seems to be the more appropriate test than Chi-squared. It is not only possible for one of these tests to be significant and the other not significant, but which is which is no doubt dependent on the characteristics of the data themselves, e.g., appropriateness of assumptions being a common explanation, another consideration the power of each test, but noise is another factor. That is, which is more significant can be, but does not have to be, due to chance alone.

Power for the Spearman's rho is almost identical to the Mann-Kendall test for detecting monotonicity in time series, also see corrigendum. The power of the Mann-Kendall test, in turn, is competitive with t-testing of slope, i.e., slight worse for linear conditions (as expected) and more powerful for nonlinearity. Power for the Friedman's test is modest and similar to Cochran Q is equivalent to the sign test's power. Thus, overall, I would expect Spearman's rho to be a better discriminator of significance than Friedman's test, in those circumstances where there is actual monotonicity.

Answer 2

You wrote

I know the Chi-squared should tell me whether there is a significant relationship at all, whereas the Spearman Rho p-value will tell me if the correlation coefficient is significant, so please don't give me answers along that line. Is it possible to have one be significant but not the other?

If you really knew the first part, you would not need to ask the question.

The chi-square statistic ignores the ordinality of the data. The rank correlation does not. Either might be what you want, and it is certainly possible for one to be significant and the other not. There are also other statistics that use ordinal data; I like the Jonckheere-Terpstra test which seems to me to look at "ordinality" most strictly.