I'm reading chapter 6 from "Machine Learning" by Tom Mitchell, 2nd edition.

It seems like the author changes in each paragraph what "D" is without saying anything, but it becomes really confussing at the section 6.7 -> Bayes Optimal Classifier (page 175).

It says that the probability of a new instance x to be classified as a value vj (from a set of values V) is:

P(vj|D) = sumatory from all hi of H( P(vj|hi)P(hi|D) )

However, it was all the time understood that D is the whole training set we use in the classifier. If we call the new instance x, that x doesn't appear anywhere in the formula so that formula, if we are rigorous, means that we get the probability that, given a training set D (and not any new instance x), we find that our classifiers say "it's vj".


1 Answer 1


$D$ represents data, not really "training data", but the data to be used to calculate posterior.

Let us go back for the basic coin example:

  • Suppose there are two types of coins (A and B). And two types of observations (H and T).
  • Different types of coin has different chance of getting head.
  • Now, suppose we have some training data, and we can get $P(\text{Data}|\text{Hypothesis})$ and $P(\text{Hypothesis})$.
  • Suppose we are running in "testing mode": randomly pick one coin and flip it and get Head (H).
  • We want to ask the coin type.

The question math representation is


So, you can see $D$ is a general representation of data, and can be "testing" data for calculating posterior.

  • $\begingroup$ So, it means that in my case, D is a different thing in P(vj|D) (D is the new instance) and P(hi|D) (D is the training data)? $\endgroup$ Commented Nov 3, 2017 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.