1
$\begingroup$

We have the equation: $P(C|X)=\sum_{i}P(C|X,M_{i})P(M_{i}|X)$

I should technically be able to prove that it works even when the total number of models is 1, so I go from the right hand side of the equation

$P(C|X,M_{1})P(M_{1}|X)=\frac{P(C,X,M_{1})P(X,M_{1})}{P(X,M_{1})P(X)}=P(C,M_{1}|X)$

But I should be able to get P(C|X). Am I missing something?

$\endgroup$
3
  • $\begingroup$ Try not removing the index $i$ when doing the calculcation. What does it mean when $i=1$? Think about it. $\endgroup$ Aug 23 '16 at 3:06
  • $\begingroup$ I don't get it? i=1 means that I am considering only the first model (and in my case the only model). $\endgroup$
    – learning
    Aug 23 '16 at 3:16
  • $\begingroup$ If you break a conditional probability into cases, then the union of all the cases you are breaking into must give you back the whole space. $\endgroup$ Aug 23 '16 at 3:19
2
$\begingroup$

You made a mistake in your calculation. You assumed $i=1$ but forgot that in order to use the law of total probability, you must have a countable partition of the sample space.

If $i=1$, then you must have $M=\Omega$ (here $\Omega$ is representing the sample space). Assuming that the collection $\{M_i\}_i$ is a partition of $\Omega$:

$$ \begin{align} \sum_i P(C|X,M_i)P(M_i|X) &= \sum_i \frac{P(C,X,M_i)}{P(X,M_i)}\cdot\frac{P(M_i,X)}{P(X)}\\ &=\sum_i \frac{P(C,M_i,X)}{P(X)}\\ &=\sum_i P(C,M_i|X)\\ &=P(C|X) \end{align} $$

where the last equality follows from the partition.

$\endgroup$
2
  • $\begingroup$ Hi, I just figured this out from your comment before you posted haha. Ultimately, it means that since there is just one model, C intersection M must just return C. Is that right? $\endgroup$
    – learning
    Aug 23 '16 at 3:19
  • $\begingroup$ Yes. There is only one model. $\endgroup$ Aug 23 '16 at 3:20
1
$\begingroup$

Note: The posterior probablity:

$P(M_i|X)= \displaystyle\frac{P(X|M_i)P(M_i)}{\sum_kP(X|M_K)P(M_K)}$

So when i=1 we get that it's equal to 1, so if there is only one M, then it will happen for sure.

More inf on model averaging: http://www.stat.colostate.edu/~jah/papers/statsci.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.