Vapnik's book talks about Fisher's discriminant analysis (which we commonly call pattern recognition) where Fisher proposes the model: There exist two categories of data distributed according to two different statistical laws $p_1(x,\alpha^*)$ and $p_1(x,\beta^*)$ (densities, belonging to parametric classes). Let the probability of occurrence of the first category of data be $q_i$ and the probability of the second category be $(1 - q_i)$. The problem is to find a decision rule that minimizes the probability of error. It says that The smallest probability of error is achieved by the decision rule that considers vector x as belonging to the first category if the probability that this vector belongs to the first category is not less than the probability that this vector belongs to the second category. This happens if the following inequality holds:

$q_i*p_1(x,\alpha^*) \geq (1 - q_i)*p_1(x,\beta^*).$

What is the reason for this?

  • $\begingroup$ That is the rule when misclassification error rates have equal cost. Under the assumption that the two class conditional densities are normal with the same variance the decision boundary is linear. You may understand this better if you read the book by Duda and Hart. $\endgroup$ May 9 '17 at 22:50
  • $\begingroup$ Can you elaborate a little on your first sentence? $\endgroup$
    – CS101
    May 10 '17 at 1:43
  • $\begingroup$ There are two ways to make errors. 1) You can classify a case from class 1 as belonging to class 2. 2) You can classify a case from class 2 as belonging to class 1. Sometimes these two types of errors are not equally important in which case you you allow more of one than the other. $\endgroup$ May 10 '17 at 2:34
  • $\begingroup$ @MichaelChernick It seems like you have a sense of how to answer this question. Perhaps you could collect your comments into an answer? $\endgroup$
    – Sycorax
    Jul 25 '18 at 1:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.