I have data coming from a real system taking measurements, and it occasionally gives a number completely wrong (ranging system with multiple returns). I'm using bayesian inference to accumulate measurements and I'd like to incorporate this chance into my model.

Now Bayes theorem gives us $$ P(H|E) = \frac{P(E|H)P(H)}{P(E)}. $$ If $T$ is the event that the observation $E$ is "true" then $$ P(E|H) = P(E|H,T)P(T) + P(E|H,!T)P(!T). $$ So the posterior becomes $$ P(H|E) = \frac{P(H)}{P(E)}\big(P(E|H,T)P(T) + P(E|H,!T)P(!T)\big). $$

In the case that $P(!T)=1$ we would like $$ P(H|E) = \frac{P(H)}{P(E)}\big(0 + P(E|H,!T)\big) =P(H), $$ that is, the posterior is equal to the prior, as we have gained no information. This gets us to $$ \frac{P(E|H)}{P(E)} = 1 $$ which says the probability of the observation is independent of the hypothesis, which is true in the model. But I don't know how to generalise it to the case where $0 < P(T) < 1$ and I think I'm missing something obvious.

Do I just set $$ P(E|H,!T) = P(E)? $$


1 Answer 1


The exact formula for the theorem that you used here is a bit misleading unless you have a neat bisection of probabilities into two classes. It's not wrong, but you need to remember to account for all the combinations.

A more general form is:

$$ p(H|E) = \frac {p(E|H)p(H)} {\sum_i { p(E|S_i) } } $$ Where $S_i$ are all the possible realizations of $E_i$ and so $H \in S$.

(an integral form would be even more general, but I do not want to wrestle with MathJax for it).

For $T_{yours} = S = \{0, 1\}$ in the case you wrote up, you wind up with exactly the formula you wrote.

If you had a weird experiment, with $S = \{0, 0, 1\}$ (e.g. you reject any outcome less than 100%), you'll get: $ p(E|H) =\\ = \frac {p(H)} {p(E)} ( (p(E|H, T=1)*(1/3) + p(E|H,T=0)*(1/3) + p(E|H,T=0)*(1/3) ) p(E|H) = \frac {p(H)} {p(E)} ( p(P(E|H, T=1)*(1/3) + p(E|H,T=0)*(2/3) )$

For a simple Bernoulli-distributed binary $T$, the parenthesized sum is: $$ (p(E|H, T=1)*p(T) + p(E|H,T=0)*(1-p(T)) $$

If $S$ is any set of $\alpha$ accept-E and $\beta$ reject-E real-valued distributions of a binary $T$, you get: $$ (p(E|H, T=1)*Beta(\alpha, \beta) + p(E|H,T=0)*(1-Beta(\alpha, \beta)) $$

If you want to go even crazier, and accept, say, $T \in \{0, 1, 2\}$, the sum expands to three terms and you'd have to draw the probabilities to multiply each by from some Categorical distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.