3
$\begingroup$

I'm just currently getting into the world of Bayesian machine learning, with a lot of frequentest stats background, and I frequently find myself limited in my modeling by my lack of knowledge about what distributions to use and when. For example, I'll have continuous data that is strongly right skewed and strictly greater than 0 by nature of some physical process. Which distribution should I use to model this?

I know I could probably use Gamma, or inverse Gaussian, chi square, or maybe a few others, but I'm not sure the implications of picking one of the above, or another distribution altogether. I know that Gamma and inverse Gaussian are in the exponential family and so have conjugate priors and are a little easier to work with, but in the age of MCMC, I'm looking for the distribution that has the best fit overall, so are there any rules of thumb for picking between these distributions? Are there certain behaviors of those distributions that are worth watching out for? Should I be taking a different approach altogether and trying to transform the data to something normal by taking logs or square roots?

Improve this question
$\endgroup$
3
  • $\begingroup$ Strictly positive is not necessarily to select distributions only positive support such as Gamma. For example, to model weight or height, they are positive but we can assume a normal distribution (with or without given $X$). For skewed data, you may consider a skewed normal distribution. I guess that it is better to think about transformation if that is possible. There a rule of thumb that simpler is better. If you could you a normal distribution, it, I think, in general, is not a bad choice. $\endgroup$
    – TrungDung
    Commented Dec 14, 2020 at 10:33
  • $\begingroup$ If your data is by its nature multiplicative (stock market prices are an example commonly given) than taking logarithms may make sense, but the point about machine learning is that you are not restricted to the convenience of theoretical distributions. $\endgroup$
    – Henry
    Commented Dec 14, 2020 at 10:33
  • $\begingroup$ Note that all chi-squared distributions are Gamma distributions but without a scale/rate parameter, so Gamma distributions are more flexible and in that sense better. $\endgroup$
    – Henry
    Commented Dec 14, 2020 at 10:33

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.