Small sample, Likert scale as independent variable

Question

I have a small sample of 25. The question of interested has a yes/no response (Do you own a car?) . I then have a series of questions with five point Likert scale responses, strongly disagree to strongly agree and I would like to explore the relationship between these questions and the yes/no car ownership question

I have created two way tabulations, my question is given the small sample size and the fact that my independent variables are measured on a Likert scale, how best should I proceed?

Most guidance that I've found seems to have the Likert scale question as the dependent variable and propose ordinal regression, but I don't think that's what I want to do here. Similarly a $\chi^{2}$ test doesn't seem appropriate, both because of the small sample and the fact that my Likert scale data is ordinal.

Answer 1

My intuition would be to use Mann-Whitney U tests checking the difference between the yes and no groups for each question. Mann-Whitney U tests can be used with ordinal variables and require fewer assumptions about the data - see this other thread.

However, if you have many questions, carrying out many different tests would inflate the risk of false positives, especially if there's correlation between the questions.

Answer 2

I've done that type of analysis a few times. I usually start by wrangling the data so that I have a column for each of the socio-demographic items, or basically anything that's not scaled data. The other columns are the Likert-style scaled items.

I will also re-scale the numeric values associated with the answers so that they make some kind of sense: I reverse the scale for the items that were reverse-coded, and will sometimes change the scale itself (e.g. 0-1-2-4 instead of 1-2-3-4) so that it fits with what the item is supposed to measure. The sample is probably too small for exploratory factor analysis or that new cool kid in Likert town, Latent class analysis, so I'll have to work with the numeric values of the answers.

Here I will split my data based on one of the non-likert items (e.g. gender) then use a statistical test to compare the sum of the answers, or of subset of answers belonging to a "construct" or "dimension" of the test. My weapon of choice for weird small sample data is the Brunner-Munzel test. There really is no situation where a Mann-Whitney U test would be better than a Brunner-Munzel (see this paper), unless you don't have access to a Brunner-Munzel in your statistical package of choice. The BM will test for stochastic dominance of group A over group B and, in it's R version, output a nice confidence interval and Common Language Effect Size. Really simple way to improve robustness of the analysis despite the small sample data. This could allow you, for instance, to compare satisfaction with distance learning between students who own or do not own a car. Do note that this will not test for a difference in means, but for the probability of random person in group A having a different score than random person in group B.

Another common choice is to dichotomize the likert data itself, e.g. agreement versus non-agreement with the statement, satisfied vs not satisfied, etc. Then you can make a nice 2 x 2 design and test for equality there. A simple test of proportions could do the trick (I think a t-test of the dichotomous variables will do exactly that). A sample research question would be "Do more students who are unhappy with distance learning own cars?"

Other fun things to try are a correlation matrix of the numerical values of the items, of the subscores, with or without the total score in the mix. It is a poor substitute for actual psychometric modeling to check test properties, but can lead to relevant results. I recommend using Pearson correlations for that.