1
$\begingroup$

I am going through some theory regarding classification using multiple decision trees. Note that a tree has probabilities of classes on each of its leaves . Here's the equation:

P(y' = $\omega$ | x', D = d) = $\Sigma_T$P(T | D = d)P(y' = $\omega$ | x', D = d, T)

Now, let us calculate P(T | D = d). We can write:

P(T | D = d) = $\alpha$P(D = d | T)P(T), using Bayes' rule. A tree comprises two parts, $\pi$, which is its structure, and $\theta$, which represents its parameters.

P(D = d | T) can be written as P(D = d | $\theta$, $\pi$) = P(D = d | $\theta$), as the tree structure is fixed. Now, let us assume an independent distribution for each leaf:

P(D = d | $\theta$) can be written as: P(D = d | $\theta_1$,$\theta_2$....$\theta_n$), where $\theta_i$ represents the probability distribution at the ith leaf.

Now, here comes the part that I cannot understand. It says:let the data d result in d1 instances in leaf l1 , d2 instances in l2 and so on. For Binomial data samples, with Beta priors on the $\theta_i$ , we have calculated this likelihood before (we have also done it for Multinomial data samples with Dirichlet priors).

P(D = d | $\theta_1$,$\theta_2$....$\theta_n$) = $\prod_{i=1}^K$$\Gamma$($s_i,t_i;a_i,b_i$). Can someone tell me how this calculate came about? I know about priors while learning Bayesian Networks, but can't relate to them somehow.

Now, it comes to calculate P($\theta$ | $\pi$) = P($\theta_1$ | $\pi$)P($\theta_2$ | $\pi$)....P($\theta_k$ | $\pi$) (due to independence). It says that that this product is simply E($\theta$). How does this become the expectation by simply multiplying the constituent probabilities?

Also, can someone refer some material on this topic? I can't seem to find it on the Internet.

Thanks!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.