# Bernoulli Trials with Bayesian Analysis

Can Bayesian analysis be used with multi-class problems?

For example, reliability analysis is usually concerned with "failed" or "not failed" classifications. My understanding is this fits well with the intent of Bayesian analysis as it pertains to Bernoulli trials.

Can this type of Bayesian analysis be extended to multi-class problems? For example, instead of failed/not-failed, can it be used for multiple states where states are defined by a gradient of numerical scores?

In reading "Doing Bayesian Data Analysis" (Kruschke) there is a statement that "All we require in this scenario is that the space of possibilities for each datums has just two possible values that are mutually exclusive". Does this mean Bayesian analysis is not appropriate for multi-class scenarios?

No, Bayesian analysis works perfectly fine with more categories than two. As long as you can write down a likelihood for the model you have in mind (e.g. softmax regression for mutually exclusive unordered categories, ordinal logistic regression for ordered categories with constant odds between categories etc.), a lot of standard Bayesian approaches will work in a relatively straightforward fashion. In fact, you will usually not have to write down the likelihood yourself, but would rather rely on some existing implementation that matches what you want to do (for some options, see below).

Of course, if your numerical scores are fine-grained enough, it could also be a reasonable approximation to treat this as a continuous data problem, when Bayesian approaches are also perfectly applicable (e.g. a Bayesian linear regression).

In some rare situations, you get nice convenient conjugate prior distribution that you can easily update. One well known case is the Binary/Binomal case, for which a $$\text{Beta}(a,b)$$ prior gets updated to $$\text{Beta}(a+y, b+(n-y))$$ after observing $$y$$ successes out of $$n$$ tries). Or for simple unordered categories, where with e.g. 4 categories $$\text{Dirichlet}(a,b,c,d)$$ gets updated to $$\text{Dirichlet}(a+y_1,b+y_2,c+y_3,d+y_4)$$ after seeing $$n$$ tries coming out as $$y_i$$ times category $$i$$).

Once you assume something more complicated than that (e.g. there are covariates or the categories are ordered), you'll probably end up using some MCMC sampler such as Stan (can be used e.g. via the R rstan library, the Python pystan package and various other tools/programming languages have their own interfaces) or one of the nice simplified interfaces for it such as - if you are using R - rstanarm, brms or stan_polr. E.g. for ordinal logistic regression stan_polr has the stan_polr function.

• Thank you, I think this makes sense. Regarding development of a likelihood function, is it possible to implement Markov state probabilities to serve as this likelihood function? Feb 10 at 15:20
• I'm not a specialist on that, but I would assume so. E.g. the Stan user guide describes how to implement hidden Markov models (mc-stan.org/docs/2_18/stan-users-guide/hmms-section.html). I'm guessing they are similar to what you mention, but don't really know. Feb 10 at 23:33

Chapter 22 of Doing Bayesian Data Analysis, 2nd Edition, is all about multinomial (multi-class) data.

• "There is a completely new chapter (Ch. 22) on multinomial logistic regression" Ok, I am working from the 1st ed. so it looks like I may need to pick up the newer version. Thank you so much for the work! It has made me stretch a bit mentally but has been incredibly helpful and the most accessible book on Bayesian analysis I've found. I was pleasantly surprised to get a response from the actual author on here. Apr 7 at 14:15