Calculation of Bayeain rule as classifier for mixture Gaussian model

Question

Here is a paper which used a bayesian classification based on Gaussian mixture model

I read many article saying that we can fit a gaussian mixture model to a data and then, based on the estimated parameter, we can easily predict the class of the new point using Bayes rule.

For the mixture model, I understand it quite good. My problem is in understanding the Bayesian rule as a classification. The Bayesian rule is:

$p(c|x) = \frac{p(c)p(x|c)}{\sum_{i=1}^{n}p(c_i)p(x|c_i)}$.

As I understand, $p(c)$ does not dependent of the class so we do not need it in our calculation.

My questions are:

1. Assume that I have a data set with 7 observations, as follows:

   1    5.1         3.5          1.4         0.2     setosa
   2    4.9         3.0          1.4         0.2     setosa
   3    4.7         3.2          1.3         0.2     setosa
   4    4.6         3.1          1.5         0.2     setosa
   5    5.0         3.6          1.4         0.2     setosa
   6    5.4         3.9          1.7         0.4     setosa
   7    4.6         3.4          1.4         0.3     setosa
   

Assume further that the prior is 0.5. Then, to calculate, ${p(c)p(x|c)}$, do I need to multiply the prior (0.5) by the joint density for each row of my data set. For example,

0.5*joint_density at (5.1 ,  3.5 , 1.4 ,  0.2 ).

Is that correct?

Then, for the denominator, do I need to sum for each row with respect to each class. For example,

0.5*joint_denstity (of the first class) at (5.1 ,  3.5 , 1.4 ,  0.2 ) + 0.5*joint_density (of the second class) at (5.1 ,  3.5 , 1.4 ,  0.2 ).

Then, the class with high posterior is chosen.

Edit based on the answer of @gunes (thank you so much).

What I try to do is:

1. Fit a Gaussian mixture model to my data (iris with two classes only).

2. Then, I need to estimate the Gaussian densities for each class.

3. Then, I need to apply a Bayesian rule for classification.

My problem is how to calculate the denominator and nominator?

If we assumed that the estimated Gaussian density for the first class is

dnorm1 and for the second class is dnorm2. These densities are estimated at each point. Then, as I understand,

I can calculate the nominator as follows:

0.5*dnrom1 (for the first class)

and 0.5*dnorm2 (for the second class).

Is that correct?

Any help, please?

Answer 1

It seems correct, but I'd like to clarify a few things:

• Seems like you're using the Iris dataset, but have two classes instead of three. Then, 0.5 is the prior of some class, e.g. setosa. Then, for the other class, whichever it might be, the prior is 1-0.5=0.5. So, you calculate the denominator (and numerator of course) correctly.
• What you refer to as joint density is actually the class conditional joint density of features, i.e. $p(\mathbf{x}|c)=p(x_1,x_2,x_3,x_4|c)$, so that multiplying it with class prior makes sense, and you get the full joint, i.e. $p(x_1,x_2,x_3,x_4,c)$.

• I'm not sure what you mean by this statement:

  As I understand, $𝑝(𝑐)$ does not dependent of the class so we do not need it in our calculation.

  Actually, $p(x)$, which is the denominator in posterior formulation doesn't depend on class; and you don't need to calculate it, since you don't need to calculate the exact posterior value if you only want to do the classification. Classification can be made via comparing $p(c_1)p(x|c_1)$ vs $p(c_2)p(x|c_2)$ and picking the class with the larger value.