Importance of sample size. Crucially, you do not reveal your sample sizes. Proportions and
averages can be quite meaningless without them. As an extreme example,
4 heads in 10 tosses of a coin is hardly evidence of a biased coin,
but 400 heads in 1000 tosses is pretty good evidence of bias.
Simulated data for illustration. Here is an experiment with simulated data with two groups of 1000
subjects each. Specifically, let's look at a 2-sample t test of
differences in 'average' Likert scores and a 2-sample test whether
two proportions in favor differ significantly.
Here are two groups of Likert scores sampled from
somewhat different populations.
set.seed(2020)
x1 = sample(1:5, 1000, rep=T, p=c(4,6,20,40,30))
x2 = sample(1:5, 1000, rep=T, p=c(5,5,15,30,45))
Pretending that Likert scores are numerical, we get a mean of 3.9
in one group of 1000 and and a mean of 4.1 in a second group of 1000.
summary(x1); sd(x1)
Min. 1st Qu. Median Mean 3rd Qu. Max.
1.0 3.0 4.0 3.9 5.0 5.0
[1] 1.042635 # sample SD
summary(x2); sd(x2)
Min. 1st Qu. Median Mean 3rd Qu. Max.
1.000 4.000 4.000 4.064 5.000 5.000
[1] 1.11855 # sample SD
Proportions in the two 'favorable' categories are about 72% and 76%,
respectively.
n1 = sum(x1 > 3); n2 = sum(x2 > 3)
n1; n2
[1] 717
[1] 758
Welch t test. A Welch 2-sample t test (which does not assume equal variances in the
two populations) to test whether the two sample means differ significantly
shows a highly significant difference (P-value about $0.001 << 0.05).$
t.test(x ~ g)
Welch Two Sample t-test
data: x by g
t = -3.3916, df = 1988.2, p-value = 0.0007086
alternative hypothesis:
true difference in means is not equal to 0
95 percent confidence interval:
-0.25883252 -0.06916748
sample estimates:
mean in group 1 mean in group 2
3.900 4.064
Test of binomial proportions. A test whether the two proportions 'in favor' differ significantly (P-value
$0.037 < 0.05).$ [With samples as large as these, no continuity correction is needed. The test assumes binomial proportions are approximately normally distributed, which they are for samples as large as 1000.]
prop.test(c(n1,n2),c(1000,1000), cor=F)
2-sample test for equality of proportions
without continuity correction
data: c(n1, n2) out of c(1000, 1000)
X-squared = 4.3416, df = 1, p-value = 0.03719
alternative hypothesis: two.sided
95 percent confidence interval:
-0.079524487 -0.002475513
sample estimates:
prop 1 prop 2
0.717 0.758
Which interpretation of data is best in your situation? So, for my simulated data, looking at Likert means seems to give very
strong evidence that the two groups differ and looking at proportions
'in favor' gives significance at the 5% level.
Do Likert data reliably behave as interval data? You may be surprised that I strongly prefer looking at the proportions
'in favor'. These proportions are unquestionable and direct results
of the ordinal Likert responses. By contrast, the 'means' require interpretation of the Likert scores as if they were numbers, which they
are not. They are ordinal categories. Whether two 'in favor' precisely
balances one 'strongly opposed' can be a topic of controversial discussion.
Over the long run, I think one does better to analyze exactly what
the data say, rather than what they may confess to after coercion.
Wilcoxon signed-rank test. I am not saying that looking at proportions 'in favor' is the very best way for you to look at your data.
A 2-sample nonparametric Wilcoxon rank sum test treats the data
as ordinal categories. Roughly speaking, it may be taken as a test whether
scores in one group may be 'shifted' relative to those in the other group.
(Some people take this as a test whether the two sample medians are
significantly different. For some data this is OK.)
Originally, the Wilcoxon test was not formulated
for use with data having many ties (as yours do), but for large samples, the version of this test in R
accommodates to ties without error messages. The (highly significant) results of this test are shown below. I think it may make good sense
to use results of the Wilcoxon rank sum test because the test treats
the data as the categorical categories they actually are.
This test asks and answers a different question than does the test of
proportions. The issue is which question is an appropriate one for your
data.
wilcox.test(x ~ g, cor=F)
Wilcoxon rank sum test
data: x by g
W = 439580, p-value = 7.483e-07
alternative hypothesis:
true location shift is not equal to 0
