Bayes Classifier - Monte Carlo sampling

Question

Assume a feature $x \in [0,2]$ and $3$ classes $\omega_i, \, i = 1,2,3$ with likelihood functions $p_1 = \frac12, p_2 = \frac34 x (2 - x), p_3 = \frac12 x $ and decision regions: $$ \begin{align} R_1 &= [0, 0.087] \\ R_2 &= [0.087, 1.5] \\ R_3 &= [1.5, 2] \end{align} $$

Calculating the expected classification error, we get:

$$ P_e = 0.4114 $$

After that, I use Monte Carlo sampling to approximate the classification error and compare it to its theoretical value.

This is the code:

N = 1000

# Prior probabilities
P1 = 0.125
P2 = 0.5
P3 = 0.375


counter1 = 0  # Misclassified instances of class 1
counter2 = 0  # Misclassified instances of class 2
counter3 = 0  # Misclassified instances of class 3


for i in range(N):
    prior = random.uniform(0, 1)

    if 0 <= prior <= P1:
        rand_choice = random.uniform(0, 2)
        if not 0 <= rand_choice <= 0.087:
            counter1 += 1
    elif P1 <= prior <= P1 + P2:
        rand_choice = random.uniform(0, 2)
        if not 0.087 <= rand_choice <= 1.5:
            counter2 += 1
    else:
        rand_choice = random.uniform(0, 2)
        if not 1.5 <= rand_choice <= 2:
            counter3 += 1

# Expected classification errors for each class
error1 = counter1 / N
error2 = counter2 / N
error3 = counter3 / N


# Total expected classification error
error = error1 + error2 + error3

print(error)

Obviously, the above code segment doesn't take into account the likelihood functions. Can someone suggest a hint on how to include these functions in the sampling?

Answer 1

The problem seems that your Monte Carlo simulation is assuming $x$ to be a uniform random variable while it is not.

After selecting a class $w_i$ you need to generate $x$ taking into account the appropriate likelihood function. To produce values for these given distributions you can use Inverse transform sampling.