2
$\begingroup$

I am performing a Multipole Decomposition Analysis on some experimental data, essentially fitting a set of experimental data with a linear combination of functions. Annoyingly these functions are not analytic nor are they orthogonal to each other. Below is an example of such a fit.

Fitting of experimental data with various distributions

My fitted parameters are the coefficients of the functions (which are in the interval [0, 1]), to obtain these parameters and their confidence intervals I use the Markov Chain Monte Carlo technique to obtain probability distributions for the parameter sets. For parameters where this distribution is symmetric or nearly symmetric extracting the parameter and a ~68% confidence interval is easy, just find the values at the 16th, 50th, and 84th percentiles and do a bit of subtraction.

However, for asymmetric distributions this is a bit trickier. Shown below is a corner plot from the one of the MCMC samplings. Marked on the single parameter probability distributions are the 16th, 50th, and 84th percentiles, while clearly for the (nearly) symmetric distributions, the best fit parameter and its errors are well determined, they aren't for the heavily skewed / asymmetric distributions. Corner Plot For a Fit

Clearly the parameter I should pick as the "fit parameter" is the position of peak of the distribution.

My question is: How should I go about finding the confidence interval, especially when the peak is close to an edge of the distribution?

Note: This question is also posted on the Mathematics stack exchange, I just realized that this might be a better forum for such a question.

$\endgroup$
5
  • $\begingroup$ Aren't (univariate) Bayesian credible intervals already plotted along the diagonal? If not, what do those dashed vertical lines mean? $\endgroup$
    – whuber
    Sep 21 '15 at 21:51
  • $\begingroup$ As I said in the question, the dashed lines are the quantiles of the distribution. $\endgroup$ Sep 21 '15 at 23:12
  • $\begingroup$ Sorry, I didn't connect that comment with those dashed lines, because it would seem that that comment completely answers your question. What is any trickier about reading off the values of lines that are asymmetrically spaced around the center compared to those that are symmetrically spaced? $\endgroup$
    – whuber
    Sep 21 '15 at 23:26
  • $\begingroup$ Because isn't it the peaks in the probability distributions that represent the most probable value of parameters? This seems to be a big point of confusion for me. Because I see a peak in a probability distribution and I think, that is the most likely value, I should use that as my fit value, is that wrong? My point is that with these asymmetric distributions, the quantiles, are not giving anything close to the peak. Some times the 0.16 quantile is close, but never the 0.5 quantile which I would ordinarily be taking as the fitted parameter. $\endgroup$ Sep 21 '15 at 23:56
  • $\begingroup$ I should say that it is certainly possible that I am deeply misunderstanding something about Bayesian statistics, I am very much a neophyte to the subject, but I have two notes. From the perspective of the known physics of the nuclei in question, the values at/near the peaks are the most physically plausible values for the parameters. From the standpoint of optimization the values at the peaks do the best job of minimizing the chi^2 function, and thus maximizing the likelihood function. $\endgroup$ Sep 22 '15 at 0:58
1
$\begingroup$

The description suggests you have MCMC samples. this leads me to expect that the model you fit was Bayesian. If that is the case then it does not make sense to look for confidence intervals for parameters. Instead, you could provide credible intervals; typically by stating the $\alpha/2$ and $(1-\alpha)/2$ empirical quantiles from your MCMC samples.

The asymmetry of the distributions does not change this. *Unless there is something quite special to say about whether your algorithm was likely to perform badly at the corners.

EDIT: the argmax of the posterior distribution is called the MAP estimate. This is what you are calling the 'fit value'. MAP estimates are great, but in Bayesian stats, it's not all about $MAP \pm 1.96 \sigma$. For instance you could easily take the mean or median of your samples rather than the mode. Basically your samples contain a lot more information than just the fit values. Consider how you would react if the hist below was of your posterior samples, would 3 be your 'fit value', or would 0? How would you summarise your samples?

pathological

Whatever prior you use, the posterior should be a probability distribution (caveat: search for 'improper priors').

Summary: you're on the right lines using quantiles of your samples (more care may be needed if you want to go beyond univariate summaries). Your issue is thinking that there's a problem with doing that; this likely caused by conceptual carry over from frequentist stats.

$\endgroup$
2
  • $\begingroup$ This confuses me, and it is entirely possible that I am misunderstanding something. But this is the posterior probability distribution, which given that I gave a uniform distribution for the prior, doesn't that mean that it is essentially probability distribution for the parameter? Thus for a given parameter isn't the peak of the distribution the most likely value and thus the fit value? From the standpoint of the physics, the peak is certainly the most credible value. What am I missing? $\endgroup$ Sep 21 '15 at 23:15
  • $\begingroup$ Thanks for the edit and explanation Bayesian stats confuse me a bit due to my frequentist 'upbringing' as it were. $\endgroup$ Sep 22 '15 at 22:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.