4
$\begingroup$

I'm interested in estimating the distributions of a few skewed datasets, for example extreme heat, and extreme rainfall.

There are many distributions that can be fit to these kinds of data, for instance, this page shows an attempt to fit multiple distributions to flood data, including Generalised Extreme Value, Generalised Pareto, Wakeby, Lognormal, and Gumbel distributions.

The fits look like this:

Generalised Extreme Value, Generalised Pareto, Wakeby, Lognormal, and Gumbel distributions fit to flood data

The page decides to use the Generalised Pareto distribution, based purely on statistical performance. There are a few problems with this that I can see: First, I can't see the underlying data, so it's possible they are fitting on only a small handful of data, and also possible that the coarseness of the histogram is hiding interesting features of the data (e.g. a down-turn near zero). They don't provide uncertainty estimates on the performance stats either, so it's not clear if the other distributions' confidence intervals overlap. It's also possible that the sample they're fitting is biased somehow, and so skewing the result towards the a particular distribution.

With the data that I'm fitting, I'm seeing quite similar results between Generalised Extreme Value and Log Normal - at some locations, the GEV fit looks better, at others, the Log Normal fit appears to do better. However, they are all always the same kind of data, and so I feel like I should always use a consistent model. As such, I would like to know:

Is there ever a purely theoretical/conceptual basis for choosing one distribution over another, assuming that the distributions perform similarly?

$\endgroup$
  • 1
    $\begingroup$ If you have a theoretical basis then I would go for that theoretical basis. The distributions in your plot have complications like predicting a non-zero possibility of a year with a maximum temperature 100°C. Obviously such event has nothing to do with the measurements and the underlying physics on which the fits are based. $\endgroup$ – Sextus Empiricus Oct 22 at 7:39
5
$\begingroup$

First I'd like to preface by saying that fitting distributions to complicated real world data doesn't really help as much as one would think. Distributions are often generated through well-defined statistical processes, and unless your data comes from a regular and well-behaved physical or mathematical process, the identity of its distribution isn't going to give you much insight.

In regards to your actual question, which I consider part of curve/model fitting in general, you should always go for the simplest model/distribution that fits your data well. By 'simple', I mean one with the fewest parameters, but also one that the general science/math community is likely to be familiar with (I am familiar with lognormal, but not with Wakeby, for instance). Personally, my intuition is to try to fit the data above with an exponential decay.

Special care needs to be taken with extrapolation, however. The example is about flood frequency, and presumably one would want to know about the relative frequency of the big but rare floods. Many of the distributions in that plot, although they behave similarly in the interval displayed, they'll behave differently in the long tail. For instance, the gumbel max decays faster than the lognormal, which decays (I believe) faster than the generalised Pareto. Your choice of distribution affects the predicted rate of occurrence for the floods that are rarer than what's in your data.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Your final paragraph is the crux of the question! If I'm fitting data with 30-50 annual maximums, then the tail is likely to be poorly represented. I would have thought that since distributions like the GEV are specifically designed for maxima estimation, then they'd be good at optimising for that tail. But I'm not sure why the log normal wouldn't be just as good, given that I don't have data to test it. I guess I could select some of the longer site datasets, and try some cross-validation with a tail-specific performance metric... But it would be nice if there were some theoretical guidance. $\endgroup$ – naught101 Oct 15 '18 at 3:04
  • 1
    $\begingroup$ Honestly, if I knew that, I wouldn't be wasting all my time on stackexchange. I suspect it's not just me either, and that guessing at the behaviour of the tail is one of those significant unsolved problems of statistics. Use of the GEV distribution presupposes your data also follows a well-understood distribution (a normal distribution I believe for Gumbel). In any case, choosing a model with few parameters still holds, as you have less danger of overfitting. The Wakeby distribution for instance has five parameters that could easily overfit and create strange behaviour in the tails. $\endgroup$ – Ingolifs Oct 15 '18 at 3:26
  • 1
    $\begingroup$ Makes sense. I guess I can be content with "I have no idea why I'm using GEV over log normal, but at least no one else does either" :D The wikipedia page says that Gumbel is good for normal or exponential distributions, so it should be OK for my data, I think. $\endgroup$ – naught101 Oct 15 '18 at 3:49
  • $\begingroup$ "one that the general science/math community is likely to be familiar with" I find this disturbing advice. But maybe I am too much of a rebel? I have made plots that got comments "that's not how we normally do it". I agree that standards are likely strong because they've had a lot development. However, when we blindly follow standards then it's hard to find better. Not using Wakeby instead of lognormal just/only because it is not familiar, is a weak argument in my point of view (of course, one needs to have a good reason to use the alternative and not just because of being fancy). $\endgroup$ – Sextus Empiricus Oct 22 at 7:33
  • 1
    $\begingroup$ @SextusEmpiricus If you're going to use an obscure distribution, you have to be sure it's for the right reasons, i.e. that it's not just showing off to everyone that you know something obscure. I've been guilty of this in the past, and can confirm people find this annoying more than they find it impressive. Sticking with tried and true 'just because' isn't ideal but neither is doing weird new stuff 'just because'. We owe it to the people less knowledgeable than us to communicate clearly. $\endgroup$ – Ingolifs Oct 22 at 8:09
2
$\begingroup$

Are your data "block maxima"? Such as, max annual values covering multiple years of time? Or something similar? If so, the the GEV is justified, but you might ask yourself if it is informative, since the GEV is so widely encompassing of samples from such a wide diversity of source processes. If you can narrow things down, that helps. So, if the source process is multiplicative, then you might assume a lognormal. Then, if you sample it as block maxima, the subset of the GEV to use is the Gumbel (the lognormal is the "domain of attraction" of the Gumbel). If your source process is a power-law, then block max samples will be Frechet (the power law is the domain of attraction of the Frechet).

So, bottom line, it is helpful to think about the nature of the source process.

Having said that, if you find that your data are well fitted by several candidate models, as seems to be the case, then you should stick with the model that is physically motivated. The data, however, might not be sufficient to distinguish between mulitiple plausible models. Occasionally, when confronted with such a challenge, I have chosen the model the provides a physically plausible lower statisical bound.

Confident conclusions can be difficult to obtain.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

I think I disagree with Ingolifs's answer. If you can justify, in this case on physical grounds, any one of the distributions, then that distribution is worthy of quite a bit of focus, including possibly for statistical hypothesis testing. I'm most familiar with the lognormal. If your physical system can result in data that can be envisioned as resulting from the multiplication of multiple random subprocesses, then you might choose to focus on that. I'm skeptical about blind use of the Generalized Extreme-Value distribution, since it encompasses so many subprocesses (with data sampled as block-maxima) that I don't see that it has much power for hypothesis testing. I'm presently trying to iron this particular issue out (in my mind), but those are my thoughts. Thanks.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I guess another way of putting the same question would be "Is there a physical basis for choosing distribution X" (e.g. any of the distributions I've listed above). Since the GEV is the limit of any series of IID maxima, it seems that it IS appropriate, if I can assume that my underlying data is IID and that I've used an appropriate maximum series, no? I don't know whether similar things can be said for any of the other distributions though. $\endgroup$ – naught101 Oct 21 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.