Let's say I run an A/B experiment on 1,000 rats. I give one group regular food and the other a special type of food that should help their cognition and I compare their performance at mazes.
Imagine that the difference in time to complete the maze is statistically significant (alpha < 0.01), my experiment had power greater than .8 and the confidence interval does not inclued 0.
I want to understand how this affects just male rats. I'd like to ignore all the non-male subjects and see if the result is statistically significant.
I find that there is a significant difference. Can I claim those results, despite bias I may have introduced?
