0
$\begingroup$

Assume a feature $x \in [0,2]$ and $3$ classes $\omega_i, \, i = 1,2,3$ with likelihood functions $p_1 = \frac12, p_2 = \frac34 x (2 - x), p_3 = \frac12 x $ and decision regions: $$ \begin{align} R_1 &= [0, 0.087] \\ R_2 &= [0.087, 1.5] \\ R_3 &= [1.5, 2] \end{align} $$

Calculating the expected classification error, we get:

$$ P_e = 0.4114 $$

After that, I use Monte Carlo sampling to approximate the classification error and compare it to its theoretical value.

This is the code:

N = 1000

# Prior probabilities
P1 = 0.125
P2 = 0.5
P3 = 0.375


counter1 = 0  # Misclassified instances of class 1
counter2 = 0  # Misclassified instances of class 2
counter3 = 0  # Misclassified instances of class 3


for i in range(N):
    prior = random.uniform(0, 1)

    if 0 <= prior <= P1:
        rand_choice = random.uniform(0, 2)
        if not 0 <= rand_choice <= 0.087:
            counter1 += 1
    elif P1 <= prior <= P1 + P2:
        rand_choice = random.uniform(0, 2)
        if not 0.087 <= rand_choice <= 1.5:
            counter2 += 1
    else:
        rand_choice = random.uniform(0, 2)
        if not 1.5 <= rand_choice <= 2:
            counter3 += 1

# Expected classification errors for each class
error1 = counter1 / N
error2 = counter2 / N
error3 = counter3 / N


# Total expected classification error
error = error1 + error2 + error3

print(error)

Obviously, the above code segment doesn't take into account the likelihood functions. Can someone suggest a hint on how to include these functions in the sampling?

$\endgroup$
6
  • $\begingroup$ Can you describe the formula used to calculate the expected classification error? $\endgroup$
    – jpneto
    Nov 2 '20 at 19:34
  • $\begingroup$ @jpneto Yes. It's: $$P_e=\sum_{k=1}^n \int_{\mathcal R-\mathcal R_k}P(w_k)p(x|w_k)dx$$ where $R$ is the union of all $R_k$. In this case, $n=3$. $\endgroup$
    – Paris
    Nov 2 '20 at 19:40
  • $\begingroup$ Assuming $x \sim U(0,2)$ and that it is independent of the regions (that is, $p(x|w_k)= p(x)$), that formula is in accordance with your Monte Carlo value. In any of these assumptions false? $\endgroup$
    – jpneto
    Nov 2 '20 at 20:06
  • $\begingroup$ @jpneto Hm, no. $p(x| w_k)$ is not the same for every $k$. Probably this is why the values don't agree. What change should I make to the code to take into account the likelihood functions? $\endgroup$
    – Paris
    Nov 2 '20 at 20:25
  • $\begingroup$ Well, what are the likelihoods? :-) I would suggest improving the question to properly specify the problem. $\endgroup$
    – jpneto
    Nov 2 '20 at 20:35
1
$\begingroup$

The problem seems that your Monte Carlo simulation is assuming $x$ to be a uniform random variable while it is not.

After selecting a class $w_i$ you need to generate $x$ taking into account the appropriate likelihood function. To produce values for these given distributions you can use Inverse transform sampling.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.