For my academical research, I have generated eight Twitter bots. Each bot is specialized in one subject. I have four subjects and two genders (a bot can be either male or female), and each bot has a unique subject-gender combination. So you might have a male bot that's specialized in cars.

The table below shows how many followers each bot managed to gather. So, for example, the male bot with subject 1 had 692 followers. Now, I am curious whether there's a significant difference between the number of followers that the male bots gathered, versus the ones that females managed to gather.

Subject     | Female | Male
1           | 692    | 408
2           | 504    | 456
3           | 211    | 175
4           | 210    | 162

As you can see, there's quite a big difference between the number of followers from male and female bots - the female score is consistently higher for each subject. Since the differences are quite large for three of the four subject, I don't understand why the Mann-Whitney U test states that the differences are not significant (since p=0.1562).

Could someone please explain this to me? Or am I using the wrong test in this scenario?

  • 1
    $\begingroup$ Your wording suggests that these are paired data in which case you have the wrong test altogether. $\endgroup$ – mdewey May 26 '16 at 17:17
  • $\begingroup$ @mdewey I have improved the description of my experiment, hopefully this will clear things up. $\endgroup$ – Algorithm_NL May 26 '16 at 17:53
  • $\begingroup$ Yeah definitely looks paired on subject, so not Mann Whitney. But with only 4 pairs you'll have little hope of significance with any nonparametric test; your smallest possible two tailed p-value will be 0.125. $\endgroup$ – Glen_b May 26 '16 at 18:09
  • $\begingroup$ @Glen_b What test should I use then? The data is non parametric and we're dealing with two independent groups, right? $\endgroup$ – Algorithm_NL May 26 '16 at 18:10
  • $\begingroup$ "nonparametric" is an adjective applied to methods of inference or sometimes to models, but not to data. These appear to be counts, why would you not use a parametric model that should be suitable for count data? ... to me it sounds like what you're trying to do is compare proportions within each subject. There might be a reasonable way to compare them via say logistic regression (binomial glm), for example, or perhaps a Poisson regression (among other possibilities).. $\endgroup$ – Glen_b May 26 '16 at 18:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.