I am learning about Bayes theorem in machine learning .

$p(h/D) = \frac{p(D/h)p(h)}{p(D)}$

$p(h) = $prior probability of hypothesis h

$p(D)$ = prior probability of training data D

$p(h/D)$ = probability of h given D

$p(D/h)$ = probability of D given h

I am from mathematical background , so generally I calculates probability by using sets or area . I mean

$p(h)$ = cardinality of h / cardinality of sample space


$p(h)$ = area covered by h / total area

But when comes to machine learning $h$ is hypothesis and $D$ is training data , how it has to be imagined as a set or area and what is sample space ?

$D$ = Training Data = input to machine

$h$ = hypothesis = output given by machine

That is all i know .

Another doubt is its stated prior probability of h , whats make difference between the "probability of hypothesis h " and "probability of getting hypothesis h" (Since h is hypothesis output given by machine)


The best way to think of it may be as follows:

$\Pr(D)$: This represents the probability of having observed the training data. Consider the sample space to be the set of possible sets of observed data. Each will be observed with some probability and that probability for the training set is represented by $\Pr(D)$.

$\Pr(h)$: I am not entirely sure I understand your last doubt, but the prior probability of the hypothesis is the probability ascribed to the hypothesis $h$ being true prior to drawing the sample. Perhaps you can consider the sample space as the set of possible hypotheses.

| cite | improve this answer | |
  • $\begingroup$ so $Pr(h)$ = version space / hypothesis space ? $\endgroup$ – hanugm Apr 7 '14 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.