I know this is from a comic famous for taking advantage of certain analytical tendencies, but it actually looks kind of reasonable after a few minutes of staring. Can anyone outline for me what this "modified Bayes theorem" is doing?

up vote 105 down vote accepted

Well by distributing the $P(H)$ term, we obtain $$ P(H|X) = \frac{P(X|H)P(H)}{P(X)} P(C) + P(H) [1 - P(C)], $$ which we can interpret as the Law of Total Probability applied to the event $C =$ "you are using Bayesian statistics correctly." So if you are using Bayesian statistics correctly, then you recover Bayes' law (the left fraction above) and if you aren't, then you ignore the data and just use your prior on $H$.

I suppose this is a rejoinder against the criticism that in principle Bayesians can adjust the prior to support whatever conclusion they want, whereas Bayesians would argue that this is not how Bayesian statistics actually works.

(And yes, you did successfully nerd-snipe me. I'm neither a mathematician nor a physicist though, so I'm not sure how many points I'm worth.)

  • 60
    A clever joke that's embedded in the formula above is that if you're not using Bayesian statistics correctly, your inference is completely independent of the truth. – Cliff AB Oct 16 at 2:10
  • 24
    I hope you didn't type out your answer while crossing a busy street. I'll have no part in this... – eric_kernfeld Oct 16 at 12:10
  • 6
    The sort of bayesians carricatured above isn't bayesian statisticians, they are bayesian lawyers – kjetil b halvorsen Oct 16 at 12:28
  • 4
    @CliffAB I don't know if I'd call that a clever joke or a law of nature. – eric_kernfeld Oct 16 at 15:30
  • 6
    @CLiffAB Do you mean "Your posterior (as calculated by this formula) is independent of the evidence"? – Acccumulation Oct 16 at 17:00

Believe it or not, this type of model does pop up every now and then in very serious statistical models, especially when dealing with data fusion, i.e., trying to combine inference from multiple sensors trying to make inference on a single event.

If a sensor malfunctions, it can greatly bias the inference made when trying to combine the signals from multiple sources. You can make a model more robust to this issue by including a small probability that the sensor is just transmitting random values, independent of the actual event of interest. This has the result that if 90 sensors weakly indicate $A$ is true, but 1 sensor strongly indicates $B$ is true, we should still conclude that $A$ is true (i.e., the posterior probability that this one sensor misfired becomes very high when we realize it contradicts all the other sensors). If the failure distribution is independent of the parameter we want to make inference on, then if the posterior probability that it is a failure is high, the measures from that sensor have very little effect on the posterior distribution for the parameter of interest; in fact, independence if the posterior probability of failure is 1.

Is this a general model that should considered when it comes to inference, i.e., should we replace Bayes theorem with Modified Bayes Theorem when doing Bayesian statistics? No. The reason is that "using Bayesian statistics correctly" isn't really just binary (or if it is, it's always false). Any analysis will have degrees of incorrect assumptions. In order for your conclusions to be completely independent from the data (which is implied by the formula), you need to make extremely grave errors. If "using Bayesian statistics incorrectly" at any level meant your analysis was completely independent of the truth, the use of statistics would be entirely worthless. All models are wrong but some are useful and all that.

  • 5
    I guess we got lucky in discovering the static failure mode of our sensors is one extreme or the other. Noise squashing is much harder though. It's really annoying to discover the sensor is working correctly and the value received is wrong because the wire is acting like an antenna. – Joshua Oct 16 at 19:53
  • @Joshua hopefully someday I'll have time to properly learn Kalman filtering for those kinds of situations (or maybe someone will write a brilliant SE answer that makes everything clear?). – mbrig Oct 17 at 18:58
  • @Joshua: I believe you are thinking of a much more specific model than I am. While well characterizing the failure distribution can improve inference, these can still be very helpful in the case of very vague failure distributions. For example, suppose we wanted to make inference about a parameter $\mu$ and sensor $i$ measure is $N(a_i \mu, 1)$ if it works. We could include a small probability that it is, say $t(df = 10)$ under failure. If sensor $i$ disagrees strongly with all other sensors, the posterior probability that it is a failure becomes very high. – Cliff AB Oct 18 at 15:26

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.