1
$\begingroup$

In the case of 2 class classification, the decision boundary occurs when 2 discriminant functions are equal: $$ g_1(x) = g_2(x) $$ $$ g_i(x) = p(x|w_i)P(w_i) $$ $$ p(x|w_i) = \mathcal{N}(x;\mu,\,\Sigma) $$

solving $$g_1(x) = g_2(x)$$ we get $w^T(x-x_0)$, where $w^T$ is $\mu_1 - \mu_2$ and $x_0$ is $1/2*(\mu_1 + \mu_2)-(\sigma^2(\mu_1 - \mu_2)/||\mu_1 - \mu_2||^2)*ln[P(w_1)/P(w_2)]$ If $P(w_1)=P(w_2)$ $x_0$ is halfway between the means, but if $P(w_1)\neq P(w_2)$ then $x_0$ shifts away from the more likely mean. My question is in the diagram the point to be classified is closer to the mean of class 2 than class 1, and the prior prob. of class 1 is 0.9 and prior prob. of class 2 is 0.1, in this case will the point be classified as class 1 or class 2?

enter image description here

$\endgroup$
1
$\begingroup$

You would still be classifying as the group with higher posterior probability. If $Y$ is the class membership indicator, $\pi_i$ the prior probability of class $i$, $f$ the density of the predictor $x$ and $i = 1, 2$ this equals

$$ P(Y = i \mid x) = \frac{\pi_i f(x \mid Y = i)}{f(x)} . $$

Your classification rule (after plugging in suitable estimates for the prior probabilities and conditional densities) is then $\text{arg max}_{i \in \{1, 2 \}} \pi_i f(x \mid Y = i)$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ but the point is very likely belonging to class 2 because it closer to the mean of class 2, but since it is one left side of the decision boundary (i.e., in the region of class 1) it is classified as class 1. But isn't is misclassification? $\endgroup$ – nSv23 Jul 16 '15 at 19:09
  • $\begingroup$ You need to take into account the prior probability of belonging to the different classes, not just where the data point lies with respect to the conditional distributions. If a class is unlikely a priori, then you need stronger evidence to conclude that a point came from that class. (It has to be very unlikely under the more common class.) $\endgroup$ – dsaxton Jul 16 '15 at 19:31
  • $\begingroup$ thanks, but I still don't understand, lets say class 1 is apple and class 2 orange, if we look at the picture above, we can see the decision boundary has gone past the mean of the class 2, does it mean that all the points on the left of decision boundary belong to class 1? Assuming the priors are equal, then that red point is an orange but because prior prob. of class 1 is greater than class 2, even though the class cond. pdf of that point given class 2 is greater than class cond. pdf given class1, the point is being classified as an apple right? Sorry but it is bit confusing for me... $\endgroup$ – nSv23 Jul 16 '15 at 19:59
  • 1
    $\begingroup$ The picture above is not weighting the densities according to the prior probabilities, it's only looking at the conditional distribution of $x$ given the class. Just imagine in your example that the proportion of oranges in the population is vanishingly small. Shouldn't this affect your decision to classify something as an orange? $\endgroup$ – dsaxton Jul 16 '15 at 20:09
  • 1
    $\begingroup$ We can't really talk about a point being correctly classified just based on the classification rule alone. The class membership is random, so we can only talk about whether or not our classification rule is "Bayes optimal," meaning it achieves the smallest possible error rate given the true model. Any point may be misclassified, even if we use the best possible classifier. $\endgroup$ – dsaxton Jul 17 '15 at 0:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.