I have data for two groups (i.e. samples) I wish to compare but the total sample size is small (n = 29) and strongly unbalanced (n = 22 vs n = 7).
These data are logistically difficult and expensive to collect, so while 'collect more data' as an obvious solution isn't helpful in this case.
A number of different variables were measured (departure date, arrival date, duration of migration etc.) so there are multiple tests, some of which the variances are very different (the smaller sample having higher variance).
Initially a colleague ran t-tests on these data, and some were statistically significant with P<0.001, another was not significant with P=0.069. Some samples were normally distributed, others were not. Some tests involved large departures from 'equal' variances.
I have several questions:
- are t-tests appropriate here? If not, why? Does this apply only to tests where assumptions of normality and equality of variances are satisfied?
- what is a suitable alternative(s)? Perhaps a permutation test?
- unequal variance inflates the Type I error, but how? and what effect does the small, unbalanced sample size have on Type I error?