How to compare two groups on a measure of social skills that includes 5 subscales where each subscale is number correct out of 12?

Question

I have two groups of participants. Each group contains 30 children (15 girls and 15 boys). The first group has been sampled from a population of children being raised under some kind of constitutional care, while the second group represents children living in their families.

I want to measure their social skills and compare the results of the two groups. This is how the measurement goes: each child is presented a series of 12 pictures. From what it says about a single picture, I get 5 true/false values - each value represents a single social skill; true means the child proved the particular social skill when confronted with that picture and false means it didn't.

In this way, I get 12 $\times$ 5 = 60 true/false values for each child.

So to visualize my data, I have something like this

$$ \begin{array} & & \mbox{Ch}_1 & \mbox{Ch}_2 & ... & \mbox{Ch}_{30} & \mbox{Ch}_{31} & ... & \mbox{Ch}_{60} \\ \mbox{Skill}_1 & 12 & 11 & ... & 10 & 11 & ... & 11 \\ \mbox{Skill}_2 & 9 & 10 & ... & 8 & 10 & ... & 12 \\ \mbox{Skill}_3 & 10 & 10 & ... & 9 & 11 & ... & 10 \\ \mbox{Skill}_4 & 7 & 7 & ... & 8 & 7 & ... & 7 \\ \mbox{Skill}_5 & 5 & 6 & ... & 5 & 5 & ... & 6 \\ \end{array} $$

Where an entry at $[i,j]$ denotes a number of $true$ values in the result list for the $i$-th social skill and the $j$-th child.

The children 1 to 30 are from the first group and the children 31 to 60 are from the second group.

Now, I would like, for each of the 5 social skills, to evaluate whether these two groups of children scored equally or if there was a statistically significant difference. From what I gathered from various sources (e.g. this), it seems like the two most likely usable tests for me would be the Fisher's test or the Mann-Whitney test.

So my question is basically: What kind of test would be appropriate for the given data and objective?

Answer 1

For each skill, you can perform a test of the mean difference between the groups. The most reasonable choice for the test is the Wilcoxon rank sum test (also known as the Mann-Whitney test). This is a nonparametric test and you would pick it because the responses are on an integer scale such as a Likert scale and hence their distribution will likely not be approximately normal. If the distribution did turn out to be approximately normal, the t-test could be used. In either case, you are doing 5 tests, one for each skill and therefore the p-values obtained should be adjusted for multiplicity (i.e. the true p-value should be higher for each test because you are doing five tests).

Fisher's exact test would be used if you wanted to compare the distribution across skills of the percentage of children having the skill in each group. This would be a single test and there would be no need to adjust the p-value. But for the Fisher test rejecting equality of distributions only tells you that there is a difference in the skill distribution but does not tell you which skill or skills are causing the difference.

Answer 2

A starting point would be a $5 \times 2$ mixed ANOVA with five levels of social skills (within-subjects) and two levels of group (between-subjects). The main effect of group will test whether there is an overall difference between the groups. The main effect of social skills will tell you whether the proportion of items correct overall differ across social skills. The interaction effect will tell you whether the size of group differences varies by type of social skill. If you find an interaction, you could then further explore this with analyses like analysis of simple effects, interaction contrasts, etc.

As a side point, I think the results will be more intuitive if you think about the scale as proportion correct.

Some might argue that because you only have a 13 point scale that you should not use the linear model. I think the mean would be a useful summary statistic for your purposes, and that it will most likely be an adequate approximation for your purposes.

If you are new to statistics, then you might want to start with Andy Field's discussion of mixed ANOVA in an SPSS context

Answer 3

Based on the information you provided that you would like, for each of the 5 social skills, evaluate whether these two groups of children scored equally or if there was a statistically significant difference. In answering the question , you first of all arrange your datasets as shown below according the skills and values or scores by child. I crafted an example as shown below;

Skill0ne ChildrenScores1
Group
1            11
1            12
1            10
.              .
.              .
.              .
2            6
2            4

Here, you create a variable and name it SkilloneGroup. This is obtained by assigning each value of children score to their respective groups. For example, CH1 (first child) has value 12 and he is from group one. This is assigned to group 1. Using SPSS and making Children Score as a dependent variable, the following results would be obtained for the differences between the two groups with regards to skill one

Now with the p-value, You can easily tell if there are differences between the grous.

repeat the same method to obtain differences for skill two three ...etc