# Bayes Classifier - Monte Carlo sampling

Assume a feature $$x \in [0,2]$$ and $$3$$ classes $$\omega_i, \, i = 1,2,3$$ with likelihood functions $$p_1 = \frac12, p_2 = \frac34 x (2 - x), p_3 = \frac12 x$$ and decision regions: \begin{align} R_1 &= [0, 0.087] \\ R_2 &= [0.087, 1.5] \\ R_3 &= [1.5, 2] \end{align}

Calculating the expected classification error, we get:

$$P_e = 0.4114$$

After that, I use Monte Carlo sampling to approximate the classification error and compare it to its theoretical value.

This is the code:

N = 1000

# Prior probabilities
P1 = 0.125
P2 = 0.5
P3 = 0.375

counter1 = 0  # Misclassified instances of class 1
counter2 = 0  # Misclassified instances of class 2
counter3 = 0  # Misclassified instances of class 3

for i in range(N):
prior = random.uniform(0, 1)

if 0 <= prior <= P1:
rand_choice = random.uniform(0, 2)
if not 0 <= rand_choice <= 0.087:
counter1 += 1
elif P1 <= prior <= P1 + P2:
rand_choice = random.uniform(0, 2)
if not 0.087 <= rand_choice <= 1.5:
counter2 += 1
else:
rand_choice = random.uniform(0, 2)
if not 1.5 <= rand_choice <= 2:
counter3 += 1

# Expected classification errors for each class
error1 = counter1 / N
error2 = counter2 / N
error3 = counter3 / N

# Total expected classification error
error = error1 + error2 + error3

print(error)


Obviously, the above code segment doesn't take into account the likelihood functions. Can someone suggest a hint on how to include these functions in the sampling?

• Can you describe the formula used to calculate the expected classification error? Nov 2 '20 at 19:34
• @jpneto Yes. It's: $$P_e=\sum_{k=1}^n \int_{\mathcal R-\mathcal R_k}P(w_k)p(x|w_k)dx$$ where $R$ is the union of all $R_k$. In this case, $n=3$. Nov 2 '20 at 19:40
• Assuming $x \sim U(0,2)$ and that it is independent of the regions (that is, $p(x|w_k)= p(x)$), that formula is in accordance with your Monte Carlo value. In any of these assumptions false? Nov 2 '20 at 20:06
• @jpneto Hm, no. $p(x| w_k)$ is not the same for every $k$. Probably this is why the values don't agree. What change should I make to the code to take into account the likelihood functions? Nov 2 '20 at 20:25
• Well, what are the likelihoods? :-) I would suggest improving the question to properly specify the problem. Nov 2 '20 at 20:35

The problem seems that your Monte Carlo simulation is assuming $$x$$ to be a uniform random variable while it is not.
After selecting a class $$w_i$$ you need to generate $$x$$ taking into account the appropriate likelihood function. To produce values for these given distributions you can use Inverse transform sampling.