I am comparing two ordinal variables (i.e., two independent Likert items). I've used a chi-squared test to test significance of a relationship between the two, and a Pearson correlation (I'm expecting the two variables to have a linear relationship) to test the strength. I have a sample size of 250 and I'm looking at a table running 4x5 (one ordinal item has four integers, and the other has 5). Admittedly, not all cells in the contingency table have a value of at least 5. After running the tests, I got a very significant p-value for the chi-squared (~10^-7), but the correlation coefficient is quite low (~0.2). Can someone explain to me how this makes sense? Or am I using the wrong methods? What would be a better method to use?
$\begingroup$
$\endgroup$
10
-
3$\begingroup$ As your sample size increases smaller and smaller effects become "statistically" significant. $\endgroup$– mdeweyCommented Jul 6, 2017 at 7:12
-
$\begingroup$ What is the scoring system you have used for Liberton items that are being referrred. ? What are you trying to ascertain ? Please be specific. $\endgroup$– user10619Commented Jul 8, 2017 at 9:42
-
$\begingroup$ @subhash c. davar Both are just integer values. For instance, a rating of pain from 1 to 5. $\endgroup$– ByakkoCommented Jul 10, 2017 at 14:26
-
$\begingroup$ Apparently, you have termed incorrectly the two variables as ordinal variables. Would you like to present your contingency table ? How did you classify the data ? Pearson correlation doesn't test the strength ! Your claim is invalid. The computation of Chi - square statistic is not possible with the kind of data you have! $\endgroup$– user10619Commented Jul 10, 2017 at 15:03
-
1$\begingroup$ I'm not quite sure what you mean by "test" the strength. It is true that correlation reflects the strength, and that you can do a test of significance on the correlation coefficient itself. Ordinal refers to any categorical variable that has an order, such as the Likert scale. I think you may be confusing this with a continuous integer scale, but with Likert, there is no set way to measure the differences between 1 and 2, 2 and 3, and so on. $\endgroup$– ByakkoCommented Jul 11, 2017 at 1:54
|
Show 5 more comments
2 Answers
$\begingroup$
$\endgroup$
4
Erring on the side of being pedagogical, Pearson correlation is not recommended for ordinal variables. Even after considering the Likert scales aspect of your data I would still be wary of using Pearson's because of the number of assumptions it requires.
Investigate Spearman or Kendall's correlations instead for effects like whether or not both measures are given by every person i.e. paired etc.
Another similar question that might be useful.
-
$\begingroup$ Hi, thank you for the response. I was under the assumption that Spearman was basically the same as Pearson, except that you convert the data to ranks prior to the Pearson analysis. Which in this case seems to be redundant as the ordinal variables are already discrete integer ranks?? Please let me know if I'm misunderstanding this. $\endgroup$– ByakkoCommented Jul 6, 2017 at 4:38
-
$\begingroup$ Spearman rank correlation does not assume similarity of variances. A quick scatterplot will give more clarity. In general, Pearson measures proportional increase/decrease in one measure with another. Spearman measures monotonic increase/decrease with some leeway for the shape of the data. Think of fitting a smoothing curve on the x,y scatterplot of these variables, if it's not a straight line, Spearman will be a better fit. Do not discount Kendall's. You can see the actual underlying code for these methods by just typing cor on the R console. $\endgroup$ Commented Jul 6, 2017 at 4:54
-
$\begingroup$ Thank you so much. I was using Excel before, but I'll switch to R. $\endgroup$– ByakkoCommented Jul 6, 2017 at 5:25
-
$\begingroup$ Sorry, I have two other questions. I've switched to using Spearman's, but the values are fairly similar to the Pearson's. My first question is: What is the difference between using a Chi-squared test vs the Spearman Rho's test? And my second question is: Why are the correlation coefficients so low when the Chi-squared test looks significant? $\endgroup$– ByakkoCommented Jul 6, 2017 at 14:02
$\begingroup$
$\endgroup$
1
My first question is: What is the difference between using a Chi-squared test vs the Spearman Rho's test?
The chi-squared test treats both variables $X$ and $Y$ as nominal (like colors, countries etc.) and thus, it can detect any sort of underlying relationship between $X$ and $Y$. In contrast, tests for linear resp. rank-correlations make use of the fact that the factor levels are ordered and are particularly suitable to detect linear resp. monotone underlying relationships. In your setting, it seems to make sense to condense the relationship to a correlation coefficient, so it would be more natural to provide p values associated with that measure instead of the less focused chi-squared test. But basically it is up to you.
Of course you can't just run all tests that you know and then pick the one with the smallest p value. Ideally, you already select an analysis strategy before looking at the data to avoid data snooping and to end up with reproducible conclusions.
PS: You are also free to use linear correlations instead of rank-correlations. Its test is basically the "linear-by-linear" test for association by Agresti, one of the godfathers of modern categorical data analysis. If you are interested, his famous book [1] is worth every penny. You will find it in every university library.
And my second question is: Why are the correlation coefficients so low when the Chi-squared test looks significant?
A small p value means strong evidence against the null hypothesis "no relationship between $X$ and $Y$". Depending on the sample size, a sample correlation of 0.2 can mean extremely strong evidence or, if the sample is small, not much evidence against this null hypothesis. Or in other words: the p value is not a measure of effect size.
[1] Agresti, A. (2002). Categorical Data Analysis, Second Edition. Hoboken, New Jersey: John Wiley & Sons.
answered Jul 11, 2017 at 19:51
-
$\begingroup$ A small p value means strong evidence against the null hypothesis "no relationship between X and Y. "good point". " $\endgroup$ Commented Feb 20, 2023 at 13:42