3
$\begingroup$

I have two independent groups of people. They were asked to select one out of five answers from a question. The five options are strongly disagree, disagree, neutral, agree and strongly agree.

I wanted to know if the their opinions for this question is different or not.

What I have done is numerate then as 1, 2, 3, 4 and 5. Then I used student t-test to test if their average scores are significantly different.

  1. Is this OK to use Student's t-test? I understand that the data is not normal. I did it for the simplicity of interpretation.
  2. Which would be a better model than Student's t-test?

--- Update ---

Is a Multinomial model good here?

$\endgroup$

2 Answers 2

2
$\begingroup$

You have ordinal categorical data, so it may not be appropriate to do a two-sample t test. which requires nearly-normal data. You have described sample distributions that seem far from normal. Possibly your data are not even really numerical. (Assigning numbers as category labels does not necessarily lead to truly numerical data.)

With no hint clue about the appearance of your summarized data or the number of subjects, the only ethical response is to show you artificial data for which a two-sample Wilcoxon (rank sum) test are appropriate.

Then you can judge whether your data might be sufficiently similar to use the same test.

set.seed(119)
x1 = sample(1:5, 100, rep=T, p = c(1,1,2,3,1))
table(x1)
x1
  1  2  3  4  5 
 13 16 25 34 12 

x2 = sample(1:5, 100, rep=T, p = c(1,1,2,3,4))
table(x2)
x2
 1  2  3  4  5 
 2  8 18 34 38 

From the tables it seems that sample x2 has fewer low scores and more high scores than x1.

Nonoverlapping notches in the sides of boxes in two boxplots, is often a sign of significant differences.

boxplot(x1, x2, horizontal=T, col=c("wheat","skyblue2"), notch=T)

enter image description here

Will a Wilcoxon test find that the higher scores in x2 are statistically significant? The answer is Yes, with a very small P-value.

wilcox.test(x1,x2)

    Wilcoxon rank sum test with continuity correction

data:  x1 and x2
W = 3086, p-value = 1.296e-06
alternative hypothesis: 
  true location shift is not equal to 0

When using the Wilcoxon signed-rank test to compare two samples of rather different shape, it may be unclear whether it is as simple as saying one sample has a significantly higher median.

However, empirical CDF (ECDF) plots of the two samples clearly show that x2 tends to 'stochastically dominate' (have larger values) then x1 The dashed blue ECDF is to the right of the brown one (thus mainly below.)

plot(ecdf(x1), col="brown", main="ECDFs of x1 (brown) and x2")
 lines(ecdf(x2), col="blue", lty="dashed", lwd=2, pch="o")

enter image description here

$\endgroup$
3
  • $\begingroup$ Thank you for your helpful and detailed explanation. Mann-Whitney U test is cool, however there is one issue. If we have more independent variables in later analysis (e.g. including confounders), I am not sure if U test can adjust them... $\endgroup$
    – Tan
    Jan 20, 2021 at 4:18
  • $\begingroup$ That is another situation, which you should describe in greater detail, for a fresh look. // My guess is Kruskal-Wallis would work instead of Wilcoxon rank sum, if you have more than two independent samples. Perhaps an overall chi-squared test on counts to start--to see if anything is going on. $\endgroup$
    – BruceET
    Jan 20, 2021 at 4:26
  • $\begingroup$ Your questions are far too vague for responsible responses. // It's like you go to the doctor and say sometimes I have sort of this funny twinge in my stomach, do you think it's anything serious? $\endgroup$
    – BruceET
    Jan 20, 2021 at 4:38
1
$\begingroup$

This is a long discussion! Basically you have to decide to do a parametric or nonparametric (=distribution-free test; data does not follow a specific distribution) test.

Question 1: It is ok to do a parametric test (2-sample t-test in your case) with skewed and nonnormal distributions if the sample size is considered: -> In a 2-samptle t test (like in your case) the sample size in each group should be at least 15 or more.

Question 2: This depends really on your data. For example if your median is better representing your data than a Mann-Whitney test (non-parametric test) would be more adequate.

$\endgroup$
3
  • $\begingroup$ Thank you. How about a Multinomial model good? I've updated my question. $\endgroup$
    – Tan
    Jan 19, 2021 at 20:01
  • $\begingroup$ In my opinion not. Do you have other independent variables? Multinomial modelling is used when the dependend variable is equivalently categorical without a meaningful order (like green hair, blue hair, brown hair, etc..). In your case you have a likert scale (orderd variable = ordinal scale = meaningful order) -> here you can use ordinal regression. Since your data is a likert scale you can either use probit regression analysis. $\endgroup$
    – TarJae
    Jan 19, 2021 at 20:30
  • $\begingroup$ Not necessarily. Multinomial can be used for ordered data, e.g. proportional odds model. Thanks for mentioning "likert scale" and "probit" regression. $\endgroup$
    – Tan
    Jan 19, 2021 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.