# Quantifying heterogeneity

I have been trying to quantify the heterogeneity of a given variable. I'd like to say that the distribution in calcite concentrations, for example, is more heterogeneous in one rock formation than another.

So far I've graphed and compared the variable distributions with overlain histograms and box plots for multiple variables, and by and large this is fine, but it takes time on the part of the reader to interpret the plots. I've been looking for more direct comparisons, say bar charts comparing an appropriate measure of sample variability in a formation e.g. the standard deviation, interquartile range, coefficient of variation, mean or median absolute deviation etc. But for various reasons I'm not satisfied with these approaches.

I can't help but think there must be a neater, succinct way of quantifying heterogeneity/variability

I'm investigating the residual error between some predicted and observed values. The residual errors for rock A are tightly distributed and for rock B they are widely non-normally distributed.

I'm trying to tie the variability observed in the error, with variability observed in other rock properties. Both rocks A and B are variable in different respects, and I’m trying to highlight the variability that might matter.

So I’ve been looking at ways of characterising the variability with a single value, say plotting all the IQR’s for each variable next to each other on a bar chart for example, where one might infer that rock A is generally more variable than rock B.

My problems is this: variables like porosity are generally always normally and tightly distributed. Permeability by contrast is always non-normal, heavily skewed, varies over 6 or 7 orders of magnitude and typically follows a power trend. For permeability I’m not happy that an IQR really captures the variability, and secondly IQRs for different variables don’t graph well next to each other given the difference in magnitude of the parent unit. I’ve looked at normalised measures of variability, but these fudge the values with mean or medians that I don’t feel reflect the data.

Perhaps there is no ideal solution for such varied distribution styles and I should just present their distributions….?

• Check out partition the variance link about Model II ANOVA Aug 8, 2013 at 19:36
• Why, specifically, are you not satisfied with the solutions you mention?
– whuber
Aug 8, 2013 at 19:48
• @whuber's question is key. "Heterogeneity" does not have a predefined meaning and you need to say more precisely what you want and why what you have tried disappoints. Concentrations are bounded but much depends on whether you are talking about very small concentrations only. Sometimes concentrations are best handled as is, sometimes on a log scale, sometimes on a logit scale; in all cases detection limits and/or reported zeros can raise problems. But without further elaboration of the question more helpful replies are difficult. Aug 9, 2013 at 2:09
• Thanks for the comments, i've included some more detail in a question edit. Aug 9, 2013 at 9:51

Porosity as I understand it is the fraction of volume that is voids, so is strictly bounded by 0 and 1 (0 and 100%), and in practice can come near 0 but rarely comes near 1.

You describe the properties of permeability well as I understand them.

As an overarching principle I would say first choose a scale on which distributions are approximately symmetric. (Approximately Gaussian or normal is a statistical ideal but less important in detail. Gaussian implies symmetric but the opposite is not true.) If you can do that, everything will become a lot clearer.

I'd expect to leave porosity values as they are.

I'd expect to have to look at permeability on a log or even a reciprocal scale. (A reciprocal scale has the simple but useful property that units are inverted too.) That's true of anything that varies over several orders of magnitude.

If you do that, you will often find that the usual kinds of measures of variability march together, for example IQR and SD will be approximately proportional, and so forth.

You might still need to worry about sensitivity of your measures of variability to outliers or very stretched tails.

With a good choice of scale, any apparent heteroscedasticity -- a related but not identical matter -- will also often be tamed to some degree.

If you can access a copy, Hoaglin, D.C., Mosteller, F. and Tukey, J.W. (Eds). 1983. Understanding robust and exploratory data analysis. New York: John Wiley remains one of the best guides in this area. I've found this book inspirational over 30 years of knowing it, and it repays occasional re-reading.