I would greatly appreciate your advice on the following problem:
I've got a large continuous dataset with lots of zeros (~95%) and I need to find the best way to test whether certain subsets of it are "interesting", ie don't seem to be drawn from the same distribution as the rest. Zero inflation comes from the fact that each data point is based a count measurement with both true and sampling zeros, but the result is continuous as it takes into account some other parameters weighted by the count (and so if the count is zero, the result is also zero).
What would be the best way to do this? I have a feeling that Wilcoxon and even brute-force permutation tests are inadequate as they get skewed by these zeros. Focussing on non-zero measurements also removes true zeros that are extremely important. Zero-inflated models for count data are well-developed, but unsuitable for my case.
I considered fitting a Tweedie distribution to the data and then fitting a glm on response=f(subset_label). Theoretically, this seems feasible, but I'm wondering whether (a) this is overkill and (b) would still implicitly assume that all zeros are sample zeros, ie would be biased in the same way (at best) as a permutation?
Intuitively, it sounds like have some kind of hierarchical design that combines a binomial statistic based on the proportion of zeros and, say, a Wilcoxon statistic computed on non-zero values (or, better still, non-zero values supplemented with a fraction of zeros based on some prior). Sounds like a Bayesian network...
Hopefully I'm not the first one having this problem, so would be very grateful if you could point me to suitable existing techniques...