I have data for ~102,000 hands of bananas. The dataset has four data points:
- Spiders present (True/False)
- Count of rotten bananas in hand
- Count of total bananas in hand
- Percent of bananas that are rotten in each hand (rotten/total)
Overall, spiders were present in 6,600 (6.5%) of banana hands. In ~102,000 hands of bananas, there are 557,000 bananas, of which 32,700 (5.9%) were rotten. In the hands where spiders were present, the rotten rate was 9.9%, and in the no spiders banana hands, the rate was 5.2%. From here, it
Hypothesis: Spiders prefer banana hands with higher rates of rotten bananas (though they do not exclusively inhabit rotten banana hands).
I would like to statistically test and prove this. Spiders-present and no-spiders present have very different numbers of groups. The data is skewed data. The vast majority of banana hands do not have any rotten bananas. Of the 6,600 spider-present banana hands, 56% do not have any rotten bananas (3,696 hands) and in the no-spider-present banana hands, 86.8% (82,807 hands) have no rotten bananas. There are lots of zeroes.
In this scenario, this is not a sample of the total world's bananas but the full population of bananas.
I calculated the following frequency table for spiders present and the rotten rate (pandas/python):
Given the high number of zeros, and the very skewed data, I was looking for nonparametric statistical tests to explain the difference in the rotten rate between the two categories (present, not present) of banana hands.
I found the Mann–Whitney U test. Running the test, I find I have incredibly small p-value (1.25e104, yes 104). The returned statistic value is astronomical compared to the descriptions, 224031919.
I believe this is the appropriate test and is valid based on these four assumptions of my data because:
- My dependent variable, rate of rotten bananas, is ordinal/continuous
- My independent variable is categorical (spiders present, spiders not present)
- There is independence of observations. Each banana occurs only once in each hand.
- My data is not normally distributed
My questions to you:
- How can I interpret these results?
- Should I be wrangling my data to make it more manageable?
- Is there a different test that you'd suggest? If yes, what is it, and why?
- Is there something else I should be considering? If yes, what is it and why?