Trying to choose between these two tests for data I've harvested from Android store. Basically, I want to see if there is any difference in the number of dangerous permissions requested by free vs paid apps. I have equal sample sizes of 1900. When I plot the data they are both highly skewed, almost like decay curves. Under student-t I understand there is an assumption of normal distribution, but not sure what has to be normally distributed, so not sure whether student t would be the right test or whether to use non-parametric mann-whitney?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
Skewness will give you trouble with the t-test, yes. You could perhaps do a Mann-whitney, but since the data are counts, you probably need a test that fits with count data.
I'd be inclined to suggest assuming something like Poisson and then conditioning on the sum (giving a binomial test) ... but since you have a mix of applications, there may be additional skewness induced by that heterogeneity.
How skew are the distributions?
How were the applications selected?
You may ultimately be best off treating the applications as a random effect.
-
$\begingroup$ the distributions are strongly skewed around 0, then fall off sharply. the applications were selected by randomly going through the letters of the alphabet, selecting 50, then killing duplicates. no effort was made to take an equal number from app categories, which is a potential bias. but other methods of selected (eg via leaderboards) also has biases..we got significant results for both students-t and mann-whitney.. $\endgroup$– piggoMar 10, 2013 at 9:54
-
1$\begingroup$ Another possibility is to do a permutation or randomization test. $\endgroup$– Glen_bMar 10, 2013 at 12:12
-
1$\begingroup$ If there were a lot of 0's (which is what I think you mean by "strongly skewed around 0") then you may need a model that accounts for that such as zero-inflated negative binomial models $\endgroup$ Mar 10, 2013 at 12:59
-
$\begingroup$ thanks! also wondering if transforming the data to a normal dist, then doing an indep students t test an option? $\endgroup$– piggoMar 10, 2013 at 19:42
-
$\begingroup$ It can be, but beware that you're no longer comparing means of the original variable. It can still pick up some more general sense of increase (as long as the transformation is monotonic) - that is the null is still 'the distributions are the same' but the alternative is some general shift toward large/smaller values. e.g. a log-transformation would mean that on the original untransformed scale you were comparing something that's effectively scale (spread) rather than mean. Beware - if there's discreteness in your data (like a lot of zeros), you can't transform that away. $\endgroup$– Glen_bMar 10, 2013 at 21:45