0
$\begingroup$

Can Bayesian analysis be used with multi-class problems?

For example, reliability analysis is usually concerned with "failed" or "not failed" classifications. My understanding is this fits well with the intent of Bayesian analysis as it pertains to Bernoulli trials.

Can this type of Bayesian analysis be extended to multi-class problems? For example, instead of failed/not-failed, can it be used for multiple states where states are defined by a gradient of numerical scores?

In reading "Doing Bayesian Data Analysis" (Kruschke) there is a statement that "All we require in this scenario is that the space of possibilities for each datums has just two possible values that are mutually exclusive". Does this mean Bayesian analysis is not appropriate for multi-class scenarios?

$\endgroup$
2
$\begingroup$

No, Bayesian analysis works perfectly fine with more categories than two. As long as you can write down a likelihood for the model you have in mind (e.g. softmax regression for mutually exclusive unordered categories, ordinal logistic regression for ordered categories with constant odds between categories etc.), a lot of standard Bayesian approaches will work in a relatively straightforward fashion. In fact, you will usually not have to write down the likelihood yourself, but would rather rely on some existing implementation that matches what you want to do (for some options, see below).

Of course, if your numerical scores are fine-grained enough, it could also be a reasonable approximation to treat this as a continuous data problem, when Bayesian approaches are also perfectly applicable (e.g. a Bayesian linear regression).

In some rare situations, you get nice convenient conjugate prior distribution that you can easily update. One well known case is the Binary/Binomal case, for which a $$\text{Beta}(a,b)$$ prior gets updated to $$\text{Beta}(a+y, b+(n-y))$$ after observing $y$ successes out of $n$ tries). Or for simple unordered categories, where with e.g. 4 categories $$\text{Dirichlet}(a,b,c,d)$$ gets updated to $$\text{Dirichlet}(a+y_1,b+y_2,c+y_3,d+y_4)$$ after seeing $n$ tries coming out as $y_i$ times category $i$).

Once you assume something more complicated than that (e.g. there are covariates or the categories are ordered), you'll probably end up using some MCMC sampler such as Stan (can be used e.g. via the R rstan library, the Python pystan package and various other tools/programming languages have their own interfaces) or one of the nice simplified interfaces for it such as - if you are using R - rstanarm, brms or stan_polr. E.g. for ordinal logistic regression stan_polr has the stan_polr function.

$\endgroup$
2
  • $\begingroup$ Thank you, I think this makes sense. Regarding development of a likelihood function, is it possible to implement Markov state probabilities to serve as this likelihood function? $\endgroup$
    – coolhand
    Feb 10 at 15:20
  • 1
    $\begingroup$ I'm not a specialist on that, but I would assume so. E.g. the Stan user guide describes how to implement hidden Markov models (mc-stan.org/docs/2_18/stan-users-guide/hmms-section.html). I'm guessing they are similar to what you mention, but don't really know. $\endgroup$
    – Björn
    Feb 10 at 23:33
1
$\begingroup$

Chapter 22 of Doing Bayesian Data Analysis, 2nd Edition, is all about multinomial (multi-class) data.

$\endgroup$
1
  • $\begingroup$ "There is a completely new chapter (Ch. 22) on multinomial logistic regression" Ok, I am working from the 1st ed. so it looks like I may need to pick up the newer version. Thank you so much for the work! It has made me stretch a bit mentally but has been incredibly helpful and the most accessible book on Bayesian analysis I've found. I was pleasantly surprised to get a response from the actual author on here. $\endgroup$
    – coolhand
    Apr 7 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.