2
$\begingroup$

This question is about how to interpret a likelihood function obtained from Bayesian methods. I will give quite a bit of background and thoughts, but the question in the end is about how to interpret a very skewed and long tailed likelihood distribution. I hope that all the physics is not distracting/too long, I just wanted to make the example well defined enough to be referred to. I am also a bit of an amateur in data analysis, just know a tiny bit of Bayesian stuff.

Quick background

Most of this does not really matter for the question, but just a quick overview of where the data comes from. In my undergraduate course (that I just finished) we had to do an experiment whose aim was to measure the mass of the $\Sigma^-$ particle (whose true mass is $1197.449 \pm 0.0030$, all masses in this post are in $MeV/c^2$) from a number of projected momentum measurements after a stationary decay. The instructions we got suggested a method to analyse the data which I though was overly simplistic, since it completely ignored individual error bars. I then came up with a Bayesian method to get a likelihood function for the mass of the particle. The method is essentially a Bayesian distribution fitting that also works for asymmetric error bars, which is significant (in my opinion) in this case.

As usual, this analysis yielded a likelihood function, in this case for the mass of the particle (see graph below).

Graph description

The graph below shows the likelihood function against the mass $m$ in $MeV/c^2$. The function is correctly normalised to integrate to unity over the finite domain. In addition I calculated a number of potentially useful parameters and put them on the plot:

  • Maximum at $m_{max}\approx 1200$
  • Mean at $m_{mean}\approx 1139$

Then I also calculated a variety of possible width measures:

  • 1$\sigma$ confidence interval (I hope that is correct terminology): the integrated probability between the red boundary is equal to the 1$\sigma$ probability.
  • The Süssmann measure, which is often used for distributions with a long tail
  • The standard deviation of a Gaussian that locally fits the distribution at the maximum

Question

This distribution has 2 problems:

  • it is very skewed
  • it has a long tail

My question is how to express the result from this distribution. For the value of the mass we would usually give $m_{max}$. The width or error is what I am concerned about. A priori I would have said that the confidence interval would represent the error well. The problem is that from observation I know that it massively underestimates how good the measurement is. As you can see $m_{max}$ is very close to the true value, which might be luck of course, but was the case in a variety of such experiments (I analysed a number of students' data). The confidence interval however has a large width compared to width.

So my observations don't quite fit the results I get from Bayesian methods. That might of course be a problem with my model in the first place, my intuition however is that this comes from the long tail of the distribution. E.g. the local Gaussian approximation (statistically probably not very useful, but to understand this distribution it is) gives a much smaller width estimate. So my question is: how to deal with such a distribution in general? And if you would like to work with the example I gave: is there anything wrong with my analysis of it or do I have to go back to my model to search for the mistake.

enter image description here

$\endgroup$
1
$\begingroup$

Heavy-tailed data, power laws, extreme value distributions and theory are a special class of models in statistics. One useful way to interpret or further analyze your data would be to estimate a tail index. While there are several sophisticated estimators such as Hill's or Pickand's methods, a straightforward heuristic is to use OLS regression on the log(rank) against log(size) with the resulting coefficient providing a proxy for the index. Once that is derived, then this value can be compared against a Tweedie distribution table to determine in which family your data's distribution belongs.

There are many, many resources available that discuss heavy-tailed data, power laws, tail indexes and the like. Here are a few suggestions:

https://en.wikipedia.org/wiki/Heavy-tailed_distribution

For tail index domains see the Tweedie table "Examples" here https://en.wikipedia.org/wiki/Tweedie_distribution

Tail index heuristic Xavier Gabaix, RANK−1/2: A SIMPLE WAY TO IMPROVE THE OLS ESTIMATION OF TAIL EXPONENTS available here ... http://www.eco.uc3m.es/temp/jbes.2009.06157.pdf

Cosma Shalizi, the CMU machine learning guru has much to say about power laws here, So you think you have a power law? http://www.stat.cmu.edu/~cshalizi/2010-10-18-Meetup.pdf

$\endgroup$
  • $\begingroup$ thanks for your answer and +1! could you maybe add a few words what such a tail index would mean and how it could explain that my confidence interval is too large (if it can)? $\endgroup$ – Wolpertinger Jul 27 '16 at 15:33
  • $\begingroup$ Just as the mean of extreme valued data is biased, confidence intervals founded in gaussian normality will also be biased, unrepresentative and, in some instances, meaningless. One of the key insights into this class of distributions is that the classic distributional moments (e.g., mean, SD, skew, etc.) can be either undefined or infinite, depending on the domain. Reversion to nonparametric and robust statistics is recommended. $\endgroup$ – Mike Hunter Jul 27 '16 at 15:38
  • $\begingroup$ i'm not familiar with nonparametric and robust statistics, but I think the "confidence interval" I gave above is not using gaussian normality, i don't fit a gaussian to it or anything. I just found the smallest interval where the probability to lie inside is ~68%. also the süssmann measure that I gave is to my knowledge a measure that works for distributions where the moments are not defined, e.g. it gives a well-defined width for Lorentzians. it is still too large however. would you be able to share any insight on that particular matter? $\endgroup$ – Wolpertinger Jul 27 '16 at 15:43
  • $\begingroup$ Interesting comment but I will defer to the more sophisticated readers on this site. $\endgroup$ – Mike Hunter Jul 27 '16 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.