Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 4 down vote favorite
1

A Rank Histogram (or Talagrand Diagram) is a neat way of measuring whether your numerical model is giving appropriate variance. It's used for weather and climate forcasting, where you only have one observational series, and many model series (an ensemble). It's described pretty clearly here. Basically, you take a handful of runs of your model, then for each timestep/gridpoint, whatever, you calculate how the observations rank relative to the model ensemble. Then you plot a histogram of those ranks. If the histogram is u-shaped, then variance is too low (obs are rank high or low too often), if the histogram looks kind of gaussian, then the variance is too high (obs rarely ranks high or low), and if the histogram is flat, then your variance is spot-on (obs have a similar variance as the ensemble).

Examples from http://www.eumetcal.org/resources/ukmeteocal/temp/msgcal/www/english/msg/ver_prob_forec/uos4b/uos4b_ko1.htm:

Example rank histograms

So the question is, which distribution should I fit to this kind of data, and why? The latter part of the question is more important, because I think that the correct distribution is the Beta-binomial distribution, but I'm unfamiliar with this area.

share|improve this question
1  
The only possible correct answer to this question as generally posed is "practically any distribution." This is because the diagram is a tool to compare two arbitrary distributions: "observations" and the "model ensemble." They could differ in literally any fashion. By asking such a question, then, you appear to presuppose some kind of connection between the model and the observations. This is a fair assumption, but the nature of the connection depends on the kind of model and what it models. So: what kind of model and what kinds of observations do you have in mind? – whuber♦ Jul 16 '12 at 13:43
@whuber, (I'm assuming you're referring to the note at the end): right, I should have said data set types. ie. ordinal/continuous, finite/infinite domain, etc. And more specifically, data types like ranks, odds, what ever. But I guess the answer is "practically any [ordinal, finite domain] distribution", so maybe I'm better off removing that part of the question. – naught101 Jul 17 '12 at 1:55
1  
What leads you to think that the 'correct distribution would be beta-binomial'? I see nothing that really suggests it couldn't in practice be almost anything. The examples shown might be more-or-less described by a flexible discrete distribution like a beta-binomial but that doesn't make them beta-binomial. No doubt you could make an argument that would imply a beta-binomial, though I think I don't know enough about the whole area to judge whether it's tenable. – Glen_b 2 days ago
1  
Oh, okay. That wasn't clear. As I said the circumstances aren't sufficient for me to know whether the sorts of arguments one might make are tenable, but (for example) it could come from making an argument than if (something) were held constant, there'd be a situation akin to bernoulli trials, but that thing varies. You'd then argue that the varying thing should vary $p$ in those bernoulli trials. Now if the $p$ was roughly beta (and it may be possible also to give an argument for that), you would then use a beta-binomial for the mix. ...(ctd) – Glen_b 2 days ago
1  
(ctd) ... An alternative argument might start with the beta and use say a series expansion around it which values are then ranked and then try to argue that the beta-binomial is an approximation to the resulting process (such an argument would be harder to make the details work for, obviously). Or there might be other ways to argue it. – Glen_b 2 days ago
show 6 more comments

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.