In Bayesian statistics, with my variable is Gaussian distributed and I have a conjugate prior, I can solve the posterior analytically. I can still use MCMC in the case when things are non-Gaussian so long as I nominate a distribution. But what if I don't know what the appropriate distribution is? I am working with financial data which are known to be non-Gaussian (heavier tails and skewed relative to a Normal distribution). To the best of my knowledge, the exact distribution of financial data is still up for debate in academia. What would be an alternative if I don't want to make a strong assumption on how the data is distributed?

  • 6
    $\begingroup$ You might want a non-informative prior distribution. One choice might be a normal distribution with a mean that is near a plausible population mean and a huge standard deviation. Or a gamma distribution with very small shape and rate parameters. Then the posterior distribution will depend mainly on your data. $\endgroup$
    – BruceET
    Jun 20, 2021 at 7:19
  • 1
    $\begingroup$ Following @BruceET’s comment, John Kruschke’s “Bayesian Estimation Supersedes the t-test” (BEST) paper used t-distributed priors for that purpose. (As a tangent, Kruschke is a member on here and once answered a question I had about that BEST paper!) $\endgroup$
    – Dave
    Jun 20, 2021 at 7:25
  • $\begingroup$ Thanks for your replies @BruceET and Dave, will have a read of that paper. I suppose I can choose a non-informative prior, but I'd still need to choose a likelihood distribution, which for a regression, it's likely going to be Gaussian? $\endgroup$
    – stevew
    Jun 21, 2021 at 11:02
  • $\begingroup$ Just list every computer program that can generate a probability distribution, give each one a prior of 2^-(number of bits+1), calculate the likelihood of the data for each one, then update. Simple! $\endgroup$ Jun 21, 2021 at 14:05
  • $\begingroup$ @Acccumulation In practice, the assumptions that that prior distribution makes aren't worth having. (It assumes integers are much more likely than reals, for instance, but most things you do stats on involve reals.) $\endgroup$
    – wizzwizz4
    Jun 21, 2021 at 14:09

4 Answers 4


From what you’re saying is that you want something Bayesian, but you can’t define the likelihood. For such cases there is approximate Bayesian computation (see ), where in place of likelihood you use some summary statistics.

As a side note, for using proper Bayesian analysis you don’t need to know the “exact” distribution. We nearly never do. You need to use some distribution that relatively well approximates the distribution of the data. This is how is it done in most of the statistics. We don’t use Gaussian, Poisson, etc distributions because they are the exact distributions of the observed data, but they are good enough approximations for the purpose. Same you do with loss function, you don’t use squared error because it has some deep meaning for your data, but because it works well enough.

  • 2
    $\begingroup$ To elaborate, ABC is used when you don't know the distribution of the data, but, given parameter values, you have a way to sample from it. $\endgroup$ Jun 20, 2021 at 9:16
  • 3
    $\begingroup$ @AccidentalStatistician: ABC requires a specific (generative) model as the distribution of the data (or of the summary statistic), hence one "knows" this distribution if not its density function. $\endgroup$
    – Xi'an
    Jun 20, 2021 at 13:21
  • 1
    $\begingroup$ @Xi'an Yes, thanks, that was poorly worded. There are rare examples where you have the inverse cumulative distribution function for data generation, but the density function is what is required for MCMC. $\endgroup$ Jun 20, 2021 at 17:46
  • $\begingroup$ Thanks everyone for your replies. I've never heard of ABC but from the description it sounded like it's what I'm after. Will read up on this. Thanks! $\endgroup$
    – stevew
    Jun 21, 2021 at 11:28

If the distribution of the data is unknown, the Bayesian way of handling this uncertainty is to put a prior on it. There exists a huge literature on Bayesian non-parametrics, including the Fundamentals of Nonparametric Bayesian inference by Ghosal and van der Vaart. The default priors in such settings are distributions on distributions, like Dirichlet processes. Check e.g. the webpage Tutorials on Bayesian Nonparametrics, maintained by Peter Orbanz. Here is a list of seminal papers given to our students.

Concerning MCMC, there exist MCMC algorithms that handle Bayesian nonparametrics as well. See e.g. this book by Dey et al. Check also the dirichlet-process tag on this forum.

A milder solution is to use Bayesian model averaging, that is, to list all (!) the plausible families that could fit the data, choose a reference prior on each, and use the posterior mixture for quantities of interest [common to all families].

  • 1
    $\begingroup$ Thanks @Xi'an, I have used GPs once before. I suppose I wasn't sure what to do when I can't nicely describe the priors and likelihood distributions. $\endgroup$
    – stevew
    Jun 21, 2021 at 11:18
  • 1
    $\begingroup$ @stevew I think you are placing too much emphasis on "correct" distributions; nothing is a correct distribution. There is no such thing as "exact distribution of financial data" (except in a simulation). You choose models based on features of your data that you believe are important to the analysis at hand. You can then evaluate these choices based on simulation or comparison to real world data fit. If you are not sure what features matter I would recommend Bayesian model averaging or non-parametrics as described here $\endgroup$
    – bdeonovic
    Jun 22, 2021 at 17:53
  • $\begingroup$ @bdeonovic thanks for the tip! $\endgroup$
    – stevew
    Jun 22, 2021 at 22:23

Although the other answers are ok, I think they might be overkill for yours or for many other problems.

Without changing the paradigm, if you know the distribution has heavier tails than Gaussian, you can fit t-distribution with either fixed heaviness of tails or with the heaviness of tails that are estimated from the data possibly with a prior information.

If a distribution is skewed and has heavier tails, you can fit a SHASH distribution which has four parameters parametrizing location, scale, skewness, and kurtosis of the distribution with the normal distribution as its special case.

  • $\begingroup$ I've never heard of SHASH so thanks for introducing it to me. $\endgroup$
    – stevew
    Jun 22, 2021 at 22:27

I suggest as alternative bootstrap based methods, which can be parametric and not, and works with very few assumptions and very few data (only 5-7 samples suffice); it works on non gaussian distributions and for many measures (eg., mean, etc). One can "rebuild" a distribution by bootstraping the mean, if it makes sense in the given context. The Davison1 is good for introducting the bootstrap in general.

Many papers were published on the volatility smile analysed throught bootstrap based methods [0].

  1. https://scholar.google.com/scholar?q=bootstrap+volatility+smile
  2. Davison, A.C. and Hinkley, D.V., 1997. Bootstrap methods and their application (No. 1). Cambridge university press.
  3. The Non-parametric Bootstrap as a Bayesian Model
  • $\begingroup$ thanks for your reply. Not sure why this answer was voted down but I guess the only downside of this approach is speed. I'm actually using a neural network underneath all these Bayesian stuff so a bootstrapping approach might be too slow for me. $\endgroup$
    – stevew
    Jun 22, 2021 at 22:31
  • $\begingroup$ @stevew, The speed aspect needs to be actually measured. I guess you're using gpus for NN, you can do the same with bootstrap, and it can happen in parallel wiht the NN part (on different streams). If it helped or is informative, you can upvote. $\endgroup$
    – Soleil
    Jun 22, 2021 at 22:42
  • $\begingroup$ I see. Thanks again! $\endgroup$
    – stevew
    Jun 23, 2021 at 5:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.