If I want to analyze a large sample size (N = 50.000) of continuous data ($ revenue) from an A/B test, what would then be the best way to check for normality?
Thanks in advance!
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
The comments are all worthwhile and you may not need to test for normality at all.
But if you are comparing revenue from two groups (A and B) then I would be amazed if it were normally distributed. In my experience, money variables are almost never normally distributed - they tend to have long tails to the right and to have outliers.
I would also wonder if comparing the mean revenue (which is what a t-test does) is really what you want (or, if it is wanted, if you might also want something else).
Testing normality for large samples has its own problems. Tests (like Shapiro Wilk and so on) are likely to give significant results (i.e. not normal) for trivial deviations from normality. Graphical methods (such as quantile normal plots) are better, but require some experience and judgment to read.
Given all this, why not go to quantile regression? It doesn't make assumptions about the residuals and gives you (at least optionally) more information?