1
$\begingroup$

I am having trouble with the following equation from Russel and Norvig's AI textbook:

pg 803

For the first equality, I believe they are converting to joint probability, using marginalization, and then using the chain rule. However, I am not entirely sure where they get the second equality from. When I do it, I don't see how to break up the $ P(X|d,h_i) $ term and I think that is the key issue. Could anyone help with this?

Improve this question
$\endgroup$
1
  • $\begingroup$ Welcome to CV! Take a glance at the self-study tag's wiki and consider adding it to your question. I've answered in the spirit of that tag. $\endgroup$
    – Sean Easter
    Commented May 11, 2017 at 21:43

1 Answer 1

Reset to default
0
$\begingroup$

To get from the left-most to the middle, consider:

$$P(X|\textbf{d})=\sum_iP(X, h_i|\textbf{d})$$

Tinker with that formula, and you'll get to the middle equation.

To get to the right-most, give a close read of the surrounding context to see if there are any conditional dependency assumptions.

Improve this answer
$\endgroup$
6
  • $\begingroup$ • Thanks for your answer, I think I see what you're saying. I got the first equality, but for the second equality I just want to confirm that what you're hinting at is the solution is $P(X|d,h_i) \equiv P(X|h_i)$, since $X$ is conditionally independent of $d$ given $h_i$. I recall reading something like that in another textbook, but I've forgotten where. Is there any intuition for why this is so? Something along the lines of the trials are independent/the previous trial does not affect the next trial (except only how it hints at what the hypothesis might be)? $\endgroup$
    – information_interchange
    Commented May 12, 2017 at 20:21
  • $\begingroup$ @information_interchange That is what I was getting at, and is, I think, implied by the authors writing "where we have assumed that each hypothesis determines a probability distribution over $X$." For intuition on conditional independence, think of this example. If an (unobserved) coin is heads, $X \sim N(0,1)$; tails, $X \sim N(1,1)$. Now, say that the coin is unfair and that you do not know the probability of heads, but you can observe that it does in fact turn up heads. (1/) $\endgroup$
    – Sean Easter
    Commented May 12, 2017 at 22:28
  • $\begingroup$ Would changing the probability of a heads change your understanding of the distribution of $X$? It wouldn't, because however likely or not a heads result is, you've observed what occurred and know the corresponding distribution. Helpful? $\endgroup$
    – Sean Easter
    Commented May 12, 2017 at 22:29
  • $\begingroup$ Hi Sean, thanks for the follow-up and I have been thinking about what you said. I don't quite understand what you mean by heads is $ X∼N(0,1) $ while tails is $ X ~ N(1,1)$ . How can the probability of a head/tails be represented by a Normal Distribution, which is not bounded on the [0,1] interval? Shouldn't we specify different hypotheses for the same outcome? $\endgroup$
    – information_interchange
    Commented May 23, 2017 at 18:20
  • $\begingroup$ Oh I think I see what you are saying! For example, So are you saying that if it IS heads, then X~N(0,1) , else if it IS tails, then X~N(1,1)? In that case, it makes sense: knowing the probability of getting a head is irrelevant after you already have gotten a head. I think a similar concept crops up in Bayes Networks in general and using d-separation. $\endgroup$
    – information_interchange
    Commented May 23, 2017 at 18:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.