In Bayesian statistics, with my variable is Gaussian distributed and I have a conjugate prior, I can solve the posterior analytically. I can still use MCMC in the case when things are non-Gaussian so long as I nominate a distribution. But what if I don't know what the appropriate distribution is? I am working with financial data which are known to be non-Gaussian (heavier tails and skewed relative to a Normal distribution). To the best of my knowledge, the exact distribution of financial data is still up for debate in academia. What would be an alternative if I don't want to make a strong assumption on how the data is distributed?
$\begingroup$
$\endgroup$
6
4 Answers
$\begingroup$
$\endgroup$
4
From what you’re saying is that you want something Bayesian, but you can’t define the likelihood. For such cases there is approximate Bayesian computation (see abc), where in place of likelihood you use some summary statistics.
As a side note, for using proper Bayesian analysis you don’t need to know the “exact” distribution. We nearly never do. You need to use some distribution that relatively well approximates the distribution of the data. This is how is it done in most of the statistics. We don’t use Gaussian, Poisson, etc distributions because they are the exact distributions of the observed data, but they are good enough approximations for the purpose. Same you do with loss function, you don’t use squared error because it has some deep meaning for your data, but because it works well enough.
-
2$\begingroup$ To elaborate, ABC is used when you don't know the distribution of the data, but, given parameter values, you have a way to sample from it. $\endgroup$ Commented Jun 20, 2021 at 9:16
-
3$\begingroup$ @AccidentalStatistician: ABC requires a specific (generative) model as the distribution of the data (or of the summary statistic), hence one "knows" this distribution if not its density function. $\endgroup$– Xi'anCommented Jun 20, 2021 at 13:21
-
1$\begingroup$ @Xi'an Yes, thanks, that was poorly worded. There are rare examples where you have the inverse cumulative distribution function for data generation, but the density function is what is required for MCMC. $\endgroup$ Commented Jun 20, 2021 at 17:46
-
$\begingroup$ Thanks everyone for your replies. I've never heard of
ABCbut from the description it sounded like it's what I'm after. Will read up on this. Thanks! $\endgroup$– stevewCommented Jun 21, 2021 at 11:28
$\begingroup$
$\endgroup$
3
If the distribution of the data is unknown, the Bayesian way of handling this uncertainty is to put a prior on it. There exists a huge literature on Bayesian non-parametrics, including the Fundamentals of Nonparametric Bayesian inference by Ghosal and van der Vaart. The default priors in such settings are distributions on distributions, like Dirichlet processes. Check e.g. the webpage Tutorials on Bayesian Nonparametrics, maintained by Peter Orbanz. Here is a list of seminal papers given to our students.
Concerning MCMC, there exist MCMC algorithms that handle Bayesian nonparametrics as well. See e.g. this book by Dey et al. Check also the dirichlet-process tag on this forum.
A milder solution is to use Bayesian model averaging, that is, to list all (!) the plausible families that could fit the data, choose a reference prior on each, and use the posterior mixture for quantities of interest [common to all families].
-
1$\begingroup$ Thanks @Xi'an, I have used GPs once before. I suppose I wasn't sure what to do when I can't nicely describe the priors and likelihood distributions. $\endgroup$– stevewCommented Jun 21, 2021 at 11:18
-
1$\begingroup$ @stevew I think you are placing too much emphasis on "correct" distributions; nothing is a correct distribution. There is no such thing as "exact distribution of financial data" (except in a simulation). You choose models based on features of your data that you believe are important to the analysis at hand. You can then evaluate these choices based on simulation or comparison to real world data fit. If you are not sure what features matter I would recommend Bayesian model averaging or non-parametrics as described here $\endgroup$ Commented Jun 22, 2021 at 17:53
-
$\begingroup$
$\endgroup$
1
Although the other answers are ok, I think they might be overkill for yours or for many other problems.
Without changing the paradigm, if you know the distribution has heavier tails than Gaussian, you can fit t-distribution with either fixed heaviness of tails or with the heaviness of tails that are estimated from the data possibly with a prior information.
If a distribution is skewed and has heavier tails, you can fit a SHASH distribution which has four parameters parametrizing location, scale, skewness, and kurtosis of the distribution with the normal distribution as its special case.
-
$\begingroup$ I've never heard of SHASH so thanks for introducing it to me. $\endgroup$– stevewCommented Jun 22, 2021 at 22:27
$\begingroup$
$\endgroup$
3
I suggest as alternative bootstrap based methods, which can be parametric and not, and works with very few assumptions and very few data (only 5-7 samples suffice); it works on non gaussian distributions and for many measures (eg., mean, etc). One can "rebuild" a distribution by bootstraping the mean, if it makes sense in the given context. The Davison1 is good for introducting the bootstrap in general.
Many papers were published on the volatility smile analysed throught bootstrap based methods [0].
- https://scholar.google.com/scholar?q=bootstrap+volatility+smile
- Davison, A.C. and Hinkley, D.V., 1997. Bootstrap methods and their application (No. 1). Cambridge university press.
- The Non-parametric Bootstrap as a Bayesian Model
-
$\begingroup$ thanks for your reply. Not sure why this answer was voted down but I guess the only downside of this approach is speed. I'm actually using a neural network underneath all these Bayesian stuff so a bootstrapping approach might be too slow for me. $\endgroup$– stevewCommented Jun 22, 2021 at 22:31
-
$\begingroup$ @stevew, The speed aspect needs to be actually measured. I guess you're using gpus for NN, you can do the same with bootstrap, and it can happen in parallel wiht the NN part (on different streams). If it helped or is informative, you can upvote. $\endgroup$– SoleilCommented Jun 22, 2021 at 22:42
-
2^-(number of bits+1), calculate the likelihood of the data for each one, then update. Simple! $\endgroup$