# Hypothesis testing for Pareto distributions

I wish to to some simple hypothesis testing of the form provided by T-Tests and ANOVA. However, my data is not normally distributed (it follows a Pareto distribution).

My understanding is that T-Tests make the assumption that the data is normally distributed and hence I won't be able to use them - is that correct? Is there something else I can do?

I'm trying to do some quality analysis on software defects, and am having trouble knowing where to start. One basic question I want to answer is:

Does software produced in department X have more defects than department Y?

As some background, we group changes to software as "patches", in which case the question becomes

Does the average patch from department X have more defects than department Y?

Here is a histogram of bugs / patch, N = 3700.

There is a philosophical issue of what it means to have "more" defects that I don't have a great answer to. The obvious choice for one of my limited knowledge is to compare the mean defects in each group, but as others have pointed out that's not clearly the best choice. The measure linked to by Procrastinator ($P(X<Y)$) seems like it captures my intuition well.

-
Thanks @Procrastinator! My goal is to analyze whether certain business processes lead to a decrease in defects. Defects are in general Pareto-distributed (almost all instances cause zero defects, but a few cause many), and I wish to answer questions like "does department X cause more defects than department Y?" – Xodarap Oct 30 '12 at 17:09
Then this question and the answers there might be of interest. – user10525 Oct 30 '12 at 17:11
@Procrastinator: I have updated the question. – Xodarap Oct 30 '12 at 21:20
@Xodarap I have been thinking a bit about this problem. It seems like you need a distribution that can account for $P(X=0)\gt 0$ and takes values in $(0,\infty)$ as well. For this, you need a mixture of a discrete and a continuous distribution. I have not figured out how to estimate $P(X<Y)$ when $X$ and $Y$ are random variables of this sort. I will let you know if I find out how to do it. You can try using the Mann-Whitney U statistic as a first attack to the problem. If your samples are large enough, then a nonparametric approach would perform well. – user10525 Nov 1 '12 at 14:35
Procrastinator: My worry is that I can separate bad from really bad. However, I can't separate good from really good (they both have 0 defects). If I made it dichotomous I would remove this inconsistency. My N=3700, I will try your method with some sanity checks and see what happens, thanks! – Xodarap Nov 2 '12 at 14:08
show 8 more comments