Name of a particular post-hoc statistical fallacy

Question

There is a surprisingly common statistical/logical error where someone picks an element from a large random sample that has some special properties and ascribes a unique origin to it as a result of those properties.

One famous example is the "Face on Mars," where among the many random hills and geologic formations on Mars photographed by Viking 1 spacecraft, one appears like a human face. Sober scientists had to explain to a gullible general public that this structure was not some alien artwork but that something "face-like" is bound to occur in a sufficiently large random dataset.

Other common examples are include when an image of Christ appears in unevenly toasted bread and some naive people ascribe it as a religious sign... "from heaven."

As context: Mathematician John Nash studied how to calculate a particular configuration of random points (say) would contain a particular spatial pattern (e.g., a square, or the outline of a teapot). Suppose in some deep space astrophotograph we find stars or galaxies in the pattern of the McDonald's restaurant double-arch logo. Would we conclude that there is something in the universe that deliberately placed this corporate logo? Of course not... Given enough random stars, we're BOUND to find some stars that fall into that double-arch pattern.

To believe that the McDonald's logo was "deliberately designed and placed in the heavens" would be a fallacy.

THAT's the fallacy I need named.

Answer 1

There are several related terms/concepts

• Seeing special meaning in unrelated things (like: face = meaning, unrelated = rocks in a random pile) is 'apophenia'.

  • When it it is recognising faces then it is called 'pareidolia'.

  • A special case related to statistics/randomness is the 'clustering illusion'. Clusters or patterns in random distributions are considered unlikely.

• Creating arguments that lead to a predetermined conclusion could be 'begging the question'.

• The behaviour to look for causal explanations in coincidentally occurring events is called 'synchronicity'.

• The 'look elsewhere effect' is the effect that the probability of a significant result is increased when we look multiple times for such significant result.

In the comments you state that the look elsewhere effect is what you were looking for. But that is solely the fallacy of a false expression of the probability of an event/effect occurring (by considering a single test and ignoring multiple times looking in other tests).

The assignment of a special causal meaning to such improbable/random event (which actually was probable, or even when it was improbable even when accounting for the look elsewhere effect, then it might still be a random event) is more related to apophenia and synchronicity.

Answer 2

I would interpret these as examples of the famous Texas sharpshooter fallacy. From wiki,

The Texas sharpshooter fallacy often arises when a person has a large amount of data at their disposal but only focuses on a small subset of that data. Some factor other than the one attributed may give all the elements in that subset some kind of common property (or pair of common properties, when arguing for correlation). If the person attempts to account for the likelihood of finding some subset in the large data with some common property by a factor other than its actual cause, then that person is likely committing a Texas sharpshooter fallacy.

The fallacy is characterized by a lack of a specific hypothesis prior to the gathering of data, or the formulation of a hypothesis only after data have already been gathered and examined.

This popular article on the faces in Mars discusses it as a sharpshooter fallacy.

Answer 3

This sounds to me like "data-dredging"

I would say that this is probably best understood as an example of data dredging, which occurs when a user dredges through a large amount of data to find some unusual pattern that appears to be "significant" when looked at in isolation. The examples you give involve observing a single unusual case with some unusual property of interest amongst a large number of sampled cases, and then assessing the statistical properties post hoc without proper consideration of the mechanism by which it was sampled (i.e., the fact that it was pulled from a large sample precisely because it had the unusual properties of interest).

Observing a hill that resembles a human face is unusual, but it is obviously much less unusual if you observe thousands of hills and choose the one that is most "face-like". Similarly, a piece of toast with the image of Christ is unusual, but it is much less unusual if you generate thousands of pieces of toast and then choose the one that is most "Christ-like" (or more broadly, like any other image of significance). From a statistical perspective, good practice in making inference from such cases would take account of the full sampling mechanism, not just the presence of a single unusual case. When a user ignores the full sampling mechanism and assesses a single case post hoc (as if it arose from taking a single observation at random) this is data dredging that biases their inference heavily in favour of a false positive. (If combined with a formal statistical hypothesis test we also call this "p-hacking" since the p-value of the test is "hacked" to a much lower value by ignoring the true sampling mechanism.)

Answer 4

The face on Mars is a bit different. It was a:

Hasty (or Faulty) Generalization Fallacy

This fallacy occurs when an argument is based on a body of evidence that is simply too small. 1

The so-called face on Mars was an artifact of the extremely low resolution of the pictures taken of the surface of Mars; once higher resolution pictures became available, no such feature was seen to be present on Cydonia. Below was the updated image, provided by NASA:

No pareidolia here!