0
$\begingroup$

Here is a paper which used a bayesian classification based on Gaussian mixture model

I read many article saying that we can fit a gaussian mixture model to a data and then, based on the estimated parameter, we can easily predict the class of the new point using Bayes rule.

For the mixture model, I understand it quite good. My problem is in understanding the Bayesian rule as a classification. The Bayesian rule is:

$p(c|x) = \frac{p(c)p(x|c)}{\sum_{i=1}^{n}p(c_i)p(x|c_i)}$.

As I understand, $p(c)$ does not dependent of the class so we do not need it in our calculation.

My questions are:

  1. Assume that I have a data set with 7 observations, as follows:

    1    5.1         3.5          1.4         0.2     setosa
    2    4.9         3.0          1.4         0.2     setosa
    3    4.7         3.2          1.3         0.2     setosa
    4    4.6         3.1          1.5         0.2     setosa
    5    5.0         3.6          1.4         0.2     setosa
    6    5.4         3.9          1.7         0.4     setosa
    7    4.6         3.4          1.4         0.3     setosa
    

Assume further that the prior is 0.5. Then, to calculate, ${p(c)p(x|c)}$, do I need to multiply the prior (0.5) by the joint density for each row of my data set. For example,

0.5*joint_density at (5.1 ,  3.5 , 1.4 ,  0.2 ).

Is that correct?

Then, for the denominator, do I need to sum for each row with respect to each class. For example,

0.5*joint_denstity (of the first class) at (5.1 ,  3.5 , 1.4 ,  0.2 ) + 0.5*joint_density (of the second class) at (5.1 ,  3.5 , 1.4 ,  0.2 ).

Then, the class with high posterior is chosen.

Edit based on the answer of @gunes (thank you so much).

What I try to do is:

  1. Fit a Gaussian mixture model to my data (iris with two classes only).

  2. Then, I need to estimate the Gaussian densities for each class.

  3. Then, I need to apply a Bayesian rule for classification.

My problem is how to calculate the denominator and nominator?

If we assumed that the estimated Gaussian density for the first class is

dnorm1 and for the second class is dnorm2. These densities are estimated at each point. Then, as I understand,

I can calculate the nominator as follows:

0.5*dnrom1 (for the first class)

and 0.5*dnorm2 (for the second class).

Is that correct?

Any help, please?

$\endgroup$
  • $\begingroup$ There is no reason to have $p(c)$ constant, as mixtures allow for varying weights. And I would avoid mentioning Bayesian in this framework, as this simply uses conditional probabilities and not Bayesian inference. $\endgroup$ – Xi'an May 1 at 10:41
  • $\begingroup$ @Xi'an thank you for your comment. Here, I just would like to use Bayesian rule as a prediction for a class of a new point. $\endgroup$ – Mary May 1 at 12:32
  • $\begingroup$ @Xi'an Thank you again for your comment. I would like to share with you a paper which used a bayesian classification based on gaussian mixture model. $\endgroup$ – Mary May 1 at 12:42
2
$\begingroup$

It seems correct, but I'd like to clarify a few things:

  • Seems like you're using the Iris dataset, but have two classes instead of three. Then, 0.5 is the prior of some class, e.g. setosa. Then, for the other class, whichever it might be, the prior is 1-0.5=0.5. So, you calculate the denominator (and numerator of course) correctly.
  • What you refer to as joint density is actually the class conditional joint density of features, i.e. $p(\mathbf{x}|c)=p(x_1,x_2,x_3,x_4|c)$, so that multiplying it with class prior makes sense, and you get the full joint, i.e. $p(x_1,x_2,x_3,x_4,c)$.
  • I'm not sure what you mean by this statement:

    As I understand, $𝑝(𝑐)$ does not dependent of the class so we do not need it in our calculation.

    Actually, $p(x)$, which is the denominator in posterior formulation doesn't depend on class; and you don't need to calculate it, since you don't need to calculate the exact posterior value if you only want to do the classification. Classification can be made via comparing $p(c_1)p(x|c_1)$ vs $p(c_2)p(x|c_2)$ and picking the class with the larger value.

$\endgroup$
  • $\begingroup$ Thank you so much for your answer. To accept the answer, I would like to understand some of your points. I would like to fit a gaussian mixture model to estimate the model parameters. Then, I would like to do a classification using Bayesian rule. For the joint density, I mean the density of the multivariate gaussian for each class. Then, multiply it by the prior of this class. I will edit my question to make it clear. $\endgroup$ – Mary May 1 at 9:52
  • $\begingroup$ density of the mv gauss for each class is a correct way to put it, then your calculations are correct. $\endgroup$ – gunes May 1 at 9:59
  • $\begingroup$ Thank you so much. I really do not know how to thank you. I will use dvmnorm to calculate the density. $\endgroup$ – Mary May 1 at 10:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.