XKCD's modified Bayes theorem: actually kinda reasonable?

I know this is from a comic famous for taking advantage of certain analytical tendencies, but it actually looks kind of reasonable after a few minutes of staring. Can anyone outline for me what this "modified Bayes theorem" is doing?

• explainxkcd.com/wiki/index.php/2059:_Modified_Bayes%27_Theorem the explanation from the author. – Tschallacka Oct 16 '18 at 9:49
• @Tschallacka What makes you think Randall wrote that? – kasperd Oct 16 '18 at 10:32
• @Tschallacka unless any of the authors is Randall himself, this is not the case. – SQB Oct 16 '18 at 10:34
• But shouldn't you apply Bayes theorem to P(C) to update its value in face of more evidence? – Yakk Oct 16 '18 at 19:50
• I'm pretty sure the $P(C)$ there is just a facetious addition. – Ian MacDonald Oct 17 '18 at 18:34

Well by distributing the $$P(H)$$ term, we obtain $$P(H|X) = \frac{P(X|H)P(H)}{P(X)} P(C) + P(H) [1 - P(C)],$$ which we can interpret as the Law of Total Probability applied to the event $$C =$$ "you are using Bayesian statistics correctly." So if you are using Bayesian statistics correctly, then you recover Bayes' law (the left fraction above) and if you aren't, then you ignore the data and just use your prior on $$H$$.

I suppose this is a rejoinder against the criticism that in principle Bayesians can adjust the prior to support whatever conclusion they want, whereas Bayesians would argue that this is not how Bayesian statistics actually works.

(And yes, you did successfully nerd-snipe me. I'm neither a mathematician nor a physicist though, so I'm not sure how many points I'm worth.)

• A clever joke that's embedded in the formula above is that if you're not using Bayesian statistics correctly, your inference is completely independent of the truth. – Cliff AB Oct 16 '18 at 2:10
• I hope you didn't type out your answer while crossing a busy street. I'll have no part in this... – eric_kernfeld Oct 16 '18 at 12:10
• The sort of bayesians carricatured above isn't bayesian statisticians, they are bayesian lawyers – kjetil b halvorsen Oct 16 '18 at 12:28
• @CliffAB I don't know if I'd call that a clever joke or a law of nature. – eric_kernfeld Oct 16 '18 at 15:30
• @CLiffAB Do you mean "Your posterior (as calculated by this formula) is independent of the evidence"? – Acccumulation Oct 16 '18 at 17:00

Believe it or not, this type of model does pop up every now and then in very serious statistical models, especially when dealing with data fusion, i.e., trying to combine inference from multiple sensors trying to make inference on a single event.

If a sensor malfunctions, it can greatly bias the inference made when trying to combine the signals from multiple sources. You can make a model more robust to this issue by including a small probability that the sensor is just transmitting random values, independent of the actual event of interest. This has the result that if 90 sensors weakly indicate $$A$$ is true, but 1 sensor strongly indicates $$B$$ is true, we should still conclude that $$A$$ is true (i.e., the posterior probability that this one sensor misfired becomes very high when we realize it contradicts all the other sensors). If the failure distribution is independent of the parameter we want to make inference on, then if the posterior probability that it is a failure is high, the measures from that sensor have very little effect on the posterior distribution for the parameter of interest; in fact, independence if the posterior probability of failure is 1.

Is this a general model that should considered when it comes to inference, i.e., should we replace Bayes theorem with Modified Bayes Theorem when doing Bayesian statistics? No. The reason is that "using Bayesian statistics correctly" isn't really just binary (or if it is, it's always false). Any analysis will have degrees of incorrect assumptions. In order for your conclusions to be completely independent from the data (which is implied by the formula), you need to make extremely grave errors. If "using Bayesian statistics incorrectly" at any level meant your analysis was completely independent of the truth, the use of statistics would be entirely worthless. All models are wrong but some are useful and all that.

• I guess we got lucky in discovering the static failure mode of our sensors is one extreme or the other. Noise squashing is much harder though. It's really annoying to discover the sensor is working correctly and the value received is wrong because the wire is acting like an antenna. – Joshua Oct 16 '18 at 19:53
• @Joshua hopefully someday I'll have time to properly learn Kalman filtering for those kinds of situations (or maybe someone will write a brilliant SE answer that makes everything clear?). – mbrig Oct 17 '18 at 18:58
• @Joshua: I believe you are thinking of a much more specific model than I am. While well characterizing the failure distribution can improve inference, these can still be very helpful in the case of very vague failure distributions. For example, suppose we wanted to make inference about a parameter $\mu$ and sensor $i$ measure is $N(a_i \mu, 1)$ if it works. We could include a small probability that it is, say $t(df = 10)$ under failure. If sensor $i$ disagrees strongly with all other sensors, the posterior probability that it is a failure becomes very high. – Cliff AB Oct 18 '18 at 15:26