What are some good exploratory analysis and diagnostic plots for count data? [duplicate]

Does anyone know of good reference material on exploratory analysis and diagnostic plots for count data?

• Yes, of course, assume for example I have count data with two crossed factors (say A and B) where each factor has 20 levels. So, each count is associated with one level of factor A and one level of factor B. And I am interested in plot that help visualize the data and also plots to use as diagnosis to help criticize models once they are tested. – user12397 Aug 17 '12 at 9:46
• This blog has a pretty nice example. You can also augment two barplots to show the summed counts of Factors A and B as a multi-panel graph. For diagnostics, perhaps you can change the raw count of the mosaic plot to be residual. – Penguin_Knight Aug 17 '12 at 11:33
• That's pretty cool, @Penguin_Knight. With 20 levels of each factor, though, how readable would a mosaic plot be? (I'm generally a fan of mosaic plots but haven't seen one with 400 tiles in it). The first plot at that site has 45 tiles, so this one would have tiles about 1/9 the size of those. But I don't have a better solution off hand. – Peter Flom Aug 17 '12 at 15:47

Even given your extended description of your project in the comments it is, IMO, still too broad a question to give much useful advice. Specifically for analysis of categorical data (of which your count data would seem to fall within given the description), I would perhaps suggest the work of Michael Friendly. He published the books Visualizing Categorical Data and Discrete Data Analysis with R: Visualization and Modeling Techniques for Categorical and Count Data. It is specific to analysis in SAS respectively R. Potentially 20 values though will be a stretch for some (many?) of the suggested categorical displays.

For diagnostic plots, I would suggest John Fox's Regression Diagnostics green book. It is in the context of linear regression models, but the majority of same diagnostics (or very similar ones) can be utilized for generalized linear models as well.

I suspect most advice given in general for EDA for continuous data types could be fairly easily extended to low count data. So I wouldn't be fixated on looking for resources specific to count data. It matters more for models, but the way to graphically explore/present/diagnose those models should be fairly similar.

Have a look at this book (and associated R package) M Friendly: Visualizing Categorical data http://www.amazon.com/Visualizing-Categorical-Data-Michael-Friendly/dp/1580256600

I will give some examples with R and the package vcd implementing ideas from the above book (the book has code in SAS).

First some plots for investigating count data distributions. Counting the number of successes in a fixed number of trials, there is the binomial distribution. But if the assumption of equal success probability in each trial does not hold (or independence is violated) we might get some other distribution. I will use the famous dataset of gender distributions in families in Saxony.

library(vcd)

data(Saxony) # Show a kind of "hanging rootogram"
gf0  <-  goodfit(Saxony, type = "binomial")
summary(gf0)
summary(gf0)

Goodness-of-fit test for binomial distribution

X^2 df     P(> X^2)
Likelihood Ratio 97.0065 11 6.978187e-16

plot(gf0)


This shows a hanging rootogram, a kind of histogram where the bars are hanging from the theoretical distribution curve, so deviations can all be compared with the x-axis. It is a "rootogram" because it shows square roots of frequencies, so that all deviations are on about the same scale (approximating the count in each bar with a Poisson distribution, for which the square root is a variance-stabilizing transformation).

We can observe a systematic deviation from the binomial distribution. Then the same with a Poisson model, using the famous horsekicks data.

data("HorseKicks")   # von Bortkiewicz's famous data:
gf  <-  goodfit(HorseKicks, type="poisson") # hanging rootogram:
summary(gf)

Goodness-of-fit test for poisson distribution

X^2 df  P(> X^2)
Likelihood Ratio 0.8682214  3 0.8330891


so in this case the null hypothesis of a Poisson distribution is not rejected.

plot(gf)


Contingency tables are also an example of count data, counting the occurrences in each cell of the table. We use the much discussed data of admissions at UC Berkeley.

data(UCBAdmissions)
Gender
Rejected 1493   1278


We show this as a mosaic plot:

 mosaicplot(apply(UCBAdmissions, c(1, 2), sum), main = "Student admissions at UC Berkeley")


Which gives the impression that a larger proportion of females is rejected. But is this true? Prospective students are only competing with other applicants at the same department, not with all the other prospective students in general. So maybe we should do the comparison for each department separately.

opar  <-  par(mfrow = c(2, 3), oma = c(0, 0, 2, 0))
for(i in 1:6)   mosaicplot(UCBAdmissions[,,i], xlab = "Admit", ylab = "Sex", main = paste("Department", LETTERS[i]))
mtext(expression(bold("Student admissions at UC Berkeley")), outer = TRUE, cex = 1.5)


and now the conclusion is less clear.