I have a pre-test, post-test, non-equivalent groups experiment, with a sample of 62 (control and experimental groups have 31 each). My objective is to compare the effectiveness of a model of teaching for developing vocabulary among pupils with different learning styles. The learning styles of the 31 of the experimental group are:

  • Visual = 5,
  • Auditory = 7,
  • Reading = 12,
  • Kinaesthetic = 1,
  • VA = 3,
  • VR = 1,
  • AR = 2.

What statistics should I employ to compare their performance (scores on vocabulary test)?


2 Answers 2


You could try approaching it in the following manner. Assume all variances are equal. You've got no way of proving or disproving this but in this case it's probably true that they're similar. The variance you'll care about is for the score of importance, that being, (postExp - preExp) - (postCont - preCont) (I'm assuming the number of vocabulary words learned is the measure). This will be a measure of the amount of improvement for each subject. Get the variance of this measure across all. This will be the variance you'll base any inferences on. Then get the mean of this measure for each group. Now calculate confidence intervals for each of those and plot them [sqrt(var/n)*t(df)]. You're a bit stuck for the groups with 1. You could do the calculation like you have 2. This will underestimate the true range of possible values but I'm sure you're not planning to publish this in a journal anyway. Just note a caution that there is really only one sample.

Each of the values can be compared to 0. Non overlap of the error bars with 0 suggests improvement in positive scores. The bars can overlap with each other and there may still be differences amongst the scores, you should read a bit about interpretation of that. Also, be careful of extracting meaning from comparisons of non-overlap with 0 to overlap with 0.

Unless your effect sizes are genuinely very large you really can't draw any conclusions here. You can't really estimate the effect size to any degree of accuracy. And it's highly unlikely there's a large effect size difference among any of your very small groups because they're just combinations of the others except kinaesthetic. You also might take all of the combo groups and just call them something like that and combine them. The multiple learning styles group would be VA + VR + AR (6).


Consider using ICC or kappa. See this post which provides some detail on this. Stata has functionality for kappa even when sample sizes are different.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.