I am going through some theory regarding classification using multiple decision trees. Note that a tree has probabilities of classes on each of its leaves . Here's the equation:

P(y' = $\omega$ | x', D = d) = $\Sigma_T$P(T | D = d)P(y' = $\omega$ | x', D = d, T)

Now, let us calculate P(T | D = d). We can write:

P(T | D = d) = $\alpha$P(D = d | T)P(T), using Bayes' rule. A tree comprises two parts, $\pi$, which is its structure, and $\theta$, which represents its parameters.

P(D = d | T) can be written as P(D = d | $\theta$, $\pi$) = P(D = d | $\theta$), as the tree structure is fixed. Now, let us assume an independent distribution for each leaf:

P(D = d | $\theta$) can be written as: P(D = d | $\theta_1$,$\theta_2$....$\theta_n$), where $\theta_i$ represents the probability distribution at the ith leaf.

Now, here comes the part that I cannot understand. It says:let the data d result in d1 instances in leaf l1 , d2 instances in l2 and so on. For Binomial data samples, with Beta priors on the $\theta_i$ , we have calculated this likelihood before (we have also done it for Multinomial data samples with Dirichlet priors).

P(D = d | $\theta_1$,$\theta_2$....$\theta_n$) = $\prod_{i=1}^K$$\Gamma$($s_i,t_i;a_i,b_i$). Can someone tell me how this calculate came about? I know about priors while learning Bayesian Networks, but can't relate to them somehow.

Now, it comes to calculate P($\theta$ | $\pi$) = P($\theta_1$ | $\pi$)P($\theta_2$ | $\pi$)....P($\theta_k$ | $\pi$) (due to independence). It says that that this product is simply E($\theta$). How does this become the expectation by simply multiplying the constituent probabilities?

Also, can someone refer some material on this topic? I can't seem to find it on the Internet.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.