# What hypothesis test should be used for two categorical variables in Likert scale?

I'll give an example.

Let's say I conducted a health survey in a hospital. I asked two questions. The first question is: 'I take medication to sleep well' and the second is: 'I feel unwell in the morning.' I obtained approximately 100 responses for both questions.

The responses range from 1 to 5 for both questions: (1) Never, (2) Rarely, (3) Sometimes, (4) Frequently, (5) Always. (Likert scale)

That is, I have 100 rows of responses that range from 1 to 5. An ordinal categorical variable.

In this case, I want to test if there is a relationship between taking medication to sleep well and feeling unwell in the morning. I was thinking of using the chi-square test.

My null hypothesis would be H0: the variables 'I take medication to sleep well' and 'I feel unwell in the morning' are independent. In other words, there is no association between sleeping well and feeling unwell. My alternative hypothesis H1: the variables 'I take medication to sleep well' and 'I feel unwell in the morning' are not independent. In other words, there is a significant association between taking medication and feeling unwell in the morning.

And if I conducted another survey about mental health, with question 1: 'I have nothing to wish for' and question 2: 'I have no value as a person' with the same response scale... would the test still be the same? A chi-square test?

• Chi-square isn't wrong but it ignores the ordering. What to do as well or instead divides even experienced statistical people. A good visualization can help. Commented Jul 31 at 11:19
• @NickCox what do you think about mann whitney u then? Commented Jul 31 at 11:27
• Well, it's nothing to do with association of two or more variables. Commented Jul 31 at 11:45
• Some personal context would help. Are you a student playing with data or a researcher hoping to put results in a thesis or a paper? Commented Jul 31 at 11:52
• See stats.stackexchange.com/questions/56322/… for some ideas on visualization. Commented Jul 31 at 12:19

The chi-squared test will test independence against the alternative of any kind of dependence, including a non-monotonic one. You need to decide whether you are really interested in this. The chi-squared ignores the ordinal information in the data, which has the advantage that it tests against any possible dependence, but the disadvantage that it will have poor power against specific alternatives such as a monotonic dependence (i.e., it may well not find a weak monotonic dependence that is there and that the methods listed below may find).

On the other hand one can pretend that the precise numeric values are meaningful (data are of interval scale) and use standard correlation or regression. As mentioned in another answer, this is done often, and not totally wrong in the sense that often the distributional theory underlying correlation or regression and required for testing is approximately fulfilled for such data. There may be floor or ceiling effects though, i.e., residual variances may depend on whether you are close to 1 or 5 or somewhere in the middle, which is assumed to not happen for standard correlation or regression. Whether this is a problem can't be said without good data visualisation. A consideration regarding whether such an approach makes sense is also, from my point of view, whether the questionnaire was set up in such a way that it implicitly communicates to interviewees that their answers would be evaluated numerically, for example by giving numbers with the category descriptors.

Still, as said before, such an approach would implicitly assume that the numerical scores have a more quantitative meaning than they actually have.

Potential analyses at ordinal level would be ordinal regression (although as a standard this would assume the $$x$$-variable to still be numerical), and, more easily, Spearman's rank correlation, which together with a permutation test may just be what you are looking for.

This (like all previous methods except the chi-squared) will look for monotonic relation between the variables; it will not find general non-monotonic patterns, although, as said before, you may not be interested in those anyway.

I see several issues with the example, and I will offer a suggestion.

1. Your data is essentially paired; the same subject answered the 2 questions (take meds; feel unwell). But a $$\chi^2$$ test loses all the pairing information. In fact, a $$\chi^2$$ test is not appropriate for paired data; there is an assumption that the subjects are independent betwen the 2 questions. You should find a test to see if e.g. subjects who often take their meds, also often feel unwell (or is it well?) in the morning.
2. The $$\chi^2$$ test is non-directonal, and is an omnibus test. All it will tell you is whether the answer to 1 question affects the answer to the other. It will not tell you in which way it affects it; e.g. it will not tell you if taking meds helps, or worsens how they feel in the AM.
3. Likert scores, as you noted, are ordinal categorical variables. That limits what can be done with these values. Yes, (too) many researchers do treat these as interval variables. Let's just say that there is disagreement as to whether this is sound practice. Now, a $$\chi^2$$ test keeps them as categorical, but ignores the ordinality.
4. What you seem to want to find out (?) is whether subjects who often take sleep medications (i.e. presumably sleep well) also often feel well in the morning. It is not really association, but just a correlation. Note that there may not be a "reverse" correlation for subjects who never/rarely take sleep medication; one of the reasons they do not take sleep medication may be that they do not need it; they already sleep well as is...
5. One of your Likert scales may be "upside down". If the first question is "I take meds", then the 2nd should be "I feel well in the AM" (if you want to show that one leads to the other). (unless you really want to test the opposite: taking meds leads to feeling unwell?)
6. Say a given subject sometimes takes sleep medication, and sometimes feels unwell in the AM. That could be interpreted as "most times I take the sleep medication I feel unwell", or "most times I do not take the sleep medication I feel unwell" (high degree of correlation, but in opposite directions), or it could be "I fell unwell randomly, regardless of whether I took the sleep medication or not" (complete lack of correlation). So for some Likert levels, you have no idea how to interpret the response.
7. With the given scenario and data, I most likely would deprecate to a binary test. I would group all the subjects who answered 4 or 5 to the 1st question in Group1, and the others in Group2, and then would look at the numbers of subjects in each group who answered 1 or 2 to the 2nd question (trying to see if taking sleep meds frequently is associated with feeling better in the AM). That would give me a 2x2 contingency matrix, which I would analyze with $$\chi^2$$, Fisher exact, and their variants (mid-P, Yates correction, etc...). See e.g. here for an online tool.
• Yes, I now see that there are issues with my example. I'll be direct: I have an Excel spreadsheet with questions about anxiety, depression, stress, and excessive internet use. I want to examine the relationships between these variables. There are approximately 30 columns (variables), with 20 of them on a Likert scale from 1 to 4 and 10 of them on a scale from 1 to 6. I already made some good visualizations like the heatmap and stacked bar plot and the spearman rank correlation. But now I think i need more tests. What do you suggest, sir? Commented Aug 1 at 4:31
• I would start by coming up with very specific questions to be answered (examine relationships between variables is too vague). E.g. "is high internet usage correlated with depression?". I would then look at the questions related to internet usage, and come up with a criterion to divide the subjects in 2 groups: "high internet usage" vs. "not high internet usage" (e.g. answered 3 of the 5 questions about internet with a score above the midpoint). Then look at the questions dealing with depression, and again dichotomize the answers (depressed/not depressed). And use a 2x2 contingency matrix. Commented Aug 1 at 17:11
• I will be more direct and talk about my problem. I have a questionnaire on anxiety, depression, and stress, and another on excessive internet use. I want to check if there is a relationship between anxiety, depression, stress, and excessive internet use. My responses to the anxiety, depression, and stress questionnaire range from 0-3. My responses to the internet use questionnaire range from 1-5+ Commented Aug 2 at 6:23
• In the DASS questionnaire, there are 21 questions. I summed up the 7 questions on anxiety, 7 on depression, and 7 on stress (21 questions in total). I then multiplied by 2 to align with the scale of the original DASS-42 (let's call these 3_new_features). The IAT questions (internet use) were simply summed and there were 20 questions (let's call this IAT_score). So far, I have done some graphical analyses such as a heatmap and a stacked bar plot for my 41 questions. Additionally+ Commented Aug 2 at 6:23
• I performed Spearman correlation tests (heatmap) between all my questions and also between the variables I created (3_new_features vs IAT_score). Now, I intend to perform a linear regression analysis to examine the coefficients and significance, i.e., to check if an increase in stress is associated with an increase in internet use, for example. Do you think this linear regression is a good analysis? Commented Aug 2 at 6:24