1
$\begingroup$

I have data from a survey given to a patient group and control group. Both groups completed the same exact survey (Likert scale). One group has an n of only 5 or 6, the other n was about 15.

What test should I conduct to determine if there is a significant difference between the two groups answers on the survey? Would this be an independent samples t-test? Do I need to do anything to address the difference in sample size?

To make matters worse, there is some data missing from 3 participants who have 1 or two answers missing on the survey.

Is this data even worth analyzing or is there not enough power?

Improve this question
$\endgroup$
1
  • $\begingroup$ I would say, a clssical case of click me. As everyone said: You should probably get more data. $\endgroup$
    – Henrik
    Commented Mar 20, 2013 at 11:29

2 Answers 2

Reset to default
1
$\begingroup$

With a sample size of 20, at best you could use small sample non-parametric statistics, rather than a t test. The sample size is not large enough to estimate the standard deviation for a parametric test. Frankly, your statistical power is too low for a meaningful inferential test unless you are interested in very strong effects. You could consider using a Mann-Whitney U test to compare the median scores. If at all possible, I would suggest getting more cases, particularly in the group with a smaller sample size, so you can carry out a parametric test (t-test) with adequate power.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The point about power is not quite accurately linked to parametric versus nonparametric tests. If the distributions of scores are not normal, the nonparametric (rank sum) test will have better power than the t test. $\endgroup$
    – Alexis
    Commented Jul 24, 2014 at 5:41
1
$\begingroup$

To answer you second question, you may or may not have enough power. Granted, I would wager that you do not have enough power, but it possible that you are dealing with a very very large effect. For instance, if you were comparing the height of 18 to 5 year olds, you could likely find statistical significance in this case. Give the t-test a shot and see what happens. Moreover, you can subsequently explore how much power you actually have using a program such as G*Power (http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/). Note that 80% power to reject a null hypothesis set at 0.05 is common.

If you clearly do not have enough power, and you cannot collect more data, then try some alternatives such as what StatisticsDoc Consulting is suggesting.

Edit: Also, its not always about statistical significance. Sometimes just describing this type of data can be helpful to whoever your audience is.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.