0
$\begingroup$

let's suppose I'm using a 4 point likert scale that I personally don't want to use but have to because I do not have the time to create a new one. And let's say it has 10 items. After participants have already responded, I manipulate the first two points "strongly disagree and disagree" into one point "disagree" and then agree and strongly agree into another point "agree". After this, I find an overall percentage for each item and for each group. So let's say 72% of group 1 agreed with items and 50% of group 2 agreed with items.

Would it be OK to do a T-Test to see if there is significant difference between the two percentages? If not, what would be the appropriate test to do in this scenario?

For some reason, I'm thinking something along the lines of chi square because of percentages but not totally sure why that would be, if that's correct, or how to conduct it.

Any help would be much appreciated.

Also, what would be the easiest way to statistically analyze this data to test if one group is significantly different than the other after manipulating the data the way I did?...as compared to the most acceptable way.

|cite|improve this question
$\endgroup$
  • $\begingroup$ Please don't add to your question in comments. If you want to add followup questions, edit the original or post a new question $\endgroup$ – Glen_b♦ Jul 20 '17 at 6:45
  • $\begingroup$ He probably won't see your comment. Post it under his answer and he'll be notified of your response. $\endgroup$ – Glen_b♦ Jul 20 '17 at 17:47
1
$\begingroup$

If your scale has 10 items, you can count percentage of "agree" answers for each person. You'll get set of values for group 1 (say, first person agrees with 10% of items, second with 40% and so on) and another set for group 2. You can compare these two sets with t-test. But, be sure to check assumption of this test: data should follow normal distribution in both groups (separately in group 1and in group 2!) and variances in both gropus should be equal.

You can check normality assumption with Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors or any other normality test (remember to apply it twice: to data from group 1, and then to data from group two). Homogeneity of variances can be checked with F, Bartlett test (these two require normality) or Levene test (this one does not).

If all the assumptions are met, you can do t-test. If normality holds but variances are significantly different you can apply Welch correction to t-test. Otherwise you can move to nonparametric procedures like Wilcoxon two sample test (also known as Mann-Whitney-Wilcoxon test).

Another option, which IMO is not perfect, is to go with chi-squared test. For this you have to count overall number (not percent!) of "agrees" and "disagrees" in both groups, create contingency table and run chi-squared test on it.

Why I think it is not perferct? Because it treats every single answer as it was independent from any other, which is not true. 10 answers from the same person don't have to be independent. And even shouldn't be, if you scale is reliable.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Why is using a formal hypothesis test to check normality a good idea? $\endgroup$ – Glen_b♦ Jul 20 '17 at 6:46
  • $\begingroup$ Beacuse it's formal :) QQ-plots and histograms are sometimes hard to interpret (at least for me) $\endgroup$ – Łukasz Deryło Jul 20 '17 at 6:52
  • $\begingroup$ It's a formal answer to a question we already know the answer to (count percentages are bounded and discrete, so they cannot possibly be normal -- why get the wrong answer to that question at small sample sizes when you already know the answer?) ... and it's also not even the question we want to answer. We're concerned about the impact of the non-normality we know we have (which is worse at small samples, not large samples --- but we will tend to mostly reject normality in larger samples, when it doesn't impact the test we wanted to do very much at all). ... ctd $\endgroup$ – Glen_b♦ Jul 20 '17 at 7:03
  • $\begingroup$ OK, I forgot about "10 items". You're right, we don't need normality check here. Data are not normal. But if there were plenty of items, percentages could behave like normal variable, even if they are indeed bounded. $\endgroup$ – Łukasz Deryło Jul 20 '17 at 7:07
  • $\begingroup$ ctd ... That is, we will tend to formally reject in just those situations where the amount of non-normality we have matters the least. $\endgroup$ – Glen_b♦ Jul 20 '17 at 7:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.