Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

A Rank Histogram (or Talagrand Diagram) is a neat way of measuring whether your numerical model is giving appropriate variance. It's used for weather and climate forcasting, where you only have one observational series, and many model series (an ensemble). It's described pretty clearly here. Basically, you take a handful of runs of your model, then for each timestep/gridpoint, whatever, you calculate how the observations rank relative to the model ensemble. Then you plot a histogram of those ranks. If the histogram is u-shaped, then variance is too low (obs are rank high or low too often), if the histogram looks kind of gaussian, then the variance is too high (obs rarely ranks high or low), and if the histogram is flat, then your variance is spot-on (obs have a similar variance as the ensemble).

Examples from

Example rank histograms

So the question is, which distribution should I fit to this kind of data, and why? The latter part of the question is more important, because I think that the correct distribution is the Beta-binomial distribution, but I'm unfamiliar with this area.

share|improve this question

closed as too broad by us╬Ár11852, whuber Jun 26 at 19:46

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

The only possible correct answer to this question as generally posed is "practically any distribution." This is because the diagram is a tool to compare two arbitrary distributions: "observations" and the "model ensemble." They could differ in literally any fashion. By asking such a question, then, you appear to presuppose some kind of connection between the model and the observations. This is a fair assumption, but the nature of the connection depends on the kind of model and what it models. So: what kind of model and what kinds of observations do you have in mind? – whuber Jul 16 '12 at 13:43
@whuber, (I'm assuming you're referring to the note at the end): right, I should have said data set types. ie. ordinal/continuous, finite/infinite domain, etc. And more specifically, data types like ranks, odds, what ever. But I guess the answer is "practically any [ordinal, finite domain] distribution", so maybe I'm better off removing that part of the question. – naught101 Jul 17 '12 at 1:55
What leads you to think that the 'correct distribution would be beta-binomial'? I see nothing that really suggests it couldn't in practice be almost anything. The examples shown might be more-or-less described by a flexible discrete distribution like a beta-binomial but that doesn't make them beta-binomial. No doubt you could make an argument that would imply a beta-binomial, though I think I don't know enough about the whole area to judge whether it's tenable. – Glen_b May 16 '13 at 3:07
Oh, okay. That wasn't clear. As I said the circumstances aren't sufficient for me to know whether the sorts of arguments one might make are tenable, but (for example) it could come from making an argument than if (something) were held constant, there'd be a situation akin to bernoulli trials, but that thing varies. You'd then argue that the varying thing should vary $p$ in those bernoulli trials. Now if the $p$ was roughly beta (and it may be possible also to give an argument for that), you would then use a beta-binomial for the mix. ...(ctd) – Glen_b May 16 '13 at 6:31
(ctd) ... An alternative argument might start with the beta and use say a series expansion around it which values are then ranked and then try to argue that the beta-binomial is an approximation to the resulting process (such an argument would be harder to make the details work for, obviously). Or there might be other ways to argue it. – Glen_b May 16 '13 at 6:34