How to interpret long-tailed likelihood distributions and their measurement uncertainty? This question is about how to interpret a likelihood function obtained from Bayesian methods. I will give quite a bit of background and thoughts, but the question in the end is about how to interpret a very skewed and long tailed likelihood distribution. I hope that all the physics is not distracting/too long, I just wanted to make the example well defined enough to be referred to. I am also a bit of an amateur in data analysis, just know a tiny bit of Bayesian stuff.
Quick background
Most of this does not really matter for the question, but just a quick overview of where the data comes from. In my undergraduate course (that I just finished) we had to do an experiment whose aim was to measure the mass of the $\Sigma^-$ particle (whose true mass is $1197.449 \pm 0.0030$, all masses in this post are in $MeV/c^2$) from a number of projected momentum measurements after a stationary decay. The instructions we got suggested a method to analyse the data which I though was overly simplistic, since it completely ignored individual error bars. I then came up with a Bayesian method to get a likelihood function for the mass of the particle. The method is essentially a Bayesian distribution fitting that also works for asymmetric error bars, which is significant (in my opinion) in this case.
As usual, this analysis yielded a likelihood function, in this case for the mass of the particle (see graph below).
Graph description
The graph below shows the likelihood function against the mass $m$ in $MeV/c^2$. The function is correctly normalised to integrate to unity over the finite domain. In addition I calculated a number of potentially useful parameters and put them on the plot:


*

*Maximum at $m_{max}\approx 1200$

*Mean at $m_{mean}\approx 1139$


Then I also calculated a variety of possible width measures:


*

*1$\sigma$ confidence interval (I hope that is correct terminology): the integrated probability between the red boundary is equal to the 1$\sigma$ probability.

*The Süssmann measure, which is often used for distributions with a long tail

*The standard deviation of a Gaussian that locally fits the distribution at the maximum


Question
This distribution has 2 problems:


*

*it is very skewed

*it has a long tail


My question is how to express the result from this distribution. For the value of the mass we would usually give $m_{max}$. The width or error is what I am concerned about. A priori I would have said that the confidence interval would represent the error well. The problem is that from observation I know that it massively underestimates how good the measurement is. As you can see $m_{max}$ is very close to the true value, which might be luck of course, but was the case in a variety of such experiments (I analysed a number of students' data). The confidence interval however has a large width compared to width.
So my observations don't quite fit the results I get from Bayesian methods. That might of course be a problem with my model in the first place, my intuition however is that this comes from the long tail of the distribution. E.g. the local Gaussian approximation (statistically probably not very useful, but to understand this distribution it is) gives a much smaller width estimate. So my question is: how to deal with such a distribution in general? And if you would like to work with the example I gave: is there anything wrong with my analysis of it or do I have to go back to my model to search for the mistake.

 A: Heavy-tailed data, power laws, extreme value distributions and theory are a special class of models in statistics. One useful way to interpret or further analyze your data would be to estimate a tail index. While there are several sophisticated estimators such as Hill's or Pickand's methods, a straightforward heuristic is to use OLS regression on the log(rank) against log(size) with the resulting coefficient providing a proxy for the index. Once that is derived, then this value can be compared against a Tweedie distribution table to determine in which family your data's distribution belongs.
There are many, many resources available that discuss heavy-tailed data, power laws, tail indexes and the like. Here are a few suggestions:
https://en.wikipedia.org/wiki/Heavy-tailed_distribution
For tail index domains see the Tweedie table "Examples" here https://en.wikipedia.org/wiki/Tweedie_distribution
Tail index heuristic
Xavier Gabaix, RANK−1/2: A SIMPLE WAY TO IMPROVE THE OLS ESTIMATION OF TAIL EXPONENTS available here ... http://www.eco.uc3m.es/temp/jbes.2009.06157.pdf
Cosma Shalizi, the CMU machine learning guru has much to say about power laws here, So you think you have a power law? http://www.stat.cmu.edu/~cshalizi/2010-10-18-Meetup.pdf
