How much does Mathematical Logic relate to Statistics?

1. To wit, to what extent did statistics professors study Mathematical Logic ("ML" henceforth)?

2. To what extent does Statistics use ML? How relevant is ML?

I mean Mathematical Logic in books like Hils and Loeser's A First Journey Through Logic

The book starts with a presentation of naive set theory, the theory of sets that mathematicians use on a daily basis. Each subsequent chapter presents one of the main areas of mathematical logic: first order logic and formal proofs, model theory, recursion theory, Gödel's incompleteness theorem, and, finally, the axiomatic set theory.

Chiswell and Hodges's Mathematical Logic

Alongside the practical examples, readers learn what can and can't be calculated; for example the correctness of a derivation proving a given sequent can be tested mechanically, but there is no general mechanical test for the existence of a derivation proving the given sequent. The undecidability results are proved rigorously in an optional final chapter, assuming Matiyasevich's theorem characterising the computably enumerable relations. Rigorous proofs of the adequacy and completeness proofs of the relevant logics are provided, with careful attention to the languages involved. Optional sections discuss the classification of mathematical structures by first-order theories; the required theory of cardinality is developed from scratch.

and Walicki's Introduction to Mathematical Logic

Starting with the basics of set theory, induction and computability, it covers propositional and first order logic — their syntax, reasoning systems and semantics. Soundness and completeness results for Hilbert's and Gentzen's systems are presented, along with simple decidability arguments. The general applicability of various concepts and techniques is demonstrated by highlighting their consistent reuse in different contexts.

• As a 7th semester Statistics major, my guess is that all Statistics professors had to learn the basics about proofs. You cannot escape basic proof-writing because Statistics is a field of Mathematics. The ones that specialize in Mathematical Statistics may know ML according to your "deeper" definition. For example, maybe the statisticians who study Jun Shao's "Mathematical Statistics" or Lehmann's "Testing Statistical Hypothesis" learned ML more deeply. Aug 28 '20 at 22:56
• Great question! The closest thing I've ever seen to mathematical logic applied to statistics is is Ben-David et al's 2019 paper Learnability can be undecidable. While it's more learning theory than statistics, it proves that a certain learnability problem is independent of ZFC. Hoping others will have more examples of mathematical logic in statistics/machine learning! Aug 28 '20 at 23:13
• You defined ML for this context and that's fine. Otherwise the field divides up into those who, nothing else said, would guess Machine Learning, those who would guess Maximum Likelihood, and everybody else. Aug 29 '20 at 0:42
• The axiom of choice affects probability in the sense that it’s required for the existence of non-Lebesgue measurable sets and that’s a reason for the restriction to $\sigma$-algebras instead of just using power sets. Although I’ve never seen a statistician not happily accept the axiom of choice and proceed as usual
– jld
Aug 29 '20 at 5:24
• I have to wonder if the person asking this question saw some article talking about the close relationship between statistics and ML, and assumed that meant mathematical logic rather than machine learning. Regardless, this is a more interesting question. Aug 29 '20 at 18:00

You can derive Bayesian statistics from mathematical logic. See its axiomatization in logic at

Cox, R. T. (1961). The Algebra of Probable Inference. Baltimore, MD: Johns Hopkins University Press.

I cannot answer how many statistics professors are required to study logic at a deep level.

You can find its extension at

Edwin Thompson Jaynes, Probability Theory: The Logic of Science, Cambridge University Press (2003).

You can also pick up anything on decision theory and you are back in mathematical logic combined with a utility function.

Answering this question requires a degree of generalisation that may obscure the diverse pathways through which people become statistics professors. Here I will give some broad generalisations based on my own observations of the pathways through which my own mentors and colleagues became statistics professors.

In my observation, most ---but not all--- statistics professors who work on theoretical material come directly from a mathematics background. Most have done an undergraduate degree in mathematics, and this piqued their interest in probability and statistics, leading to postgraduate work in statistics. For professors with this background, they were taught all the things in the coverage of an undergraduate mathematics degree, including exposure to naive set theory, axiomatic set theory, and the foundations of mathematics. This is usually something that they previously studied during undergraduate/postgraduate degrees, but it is not common for them to continue work in this area in their careers, and so they are usually quite rusty on this material.

Other statistics professors come from an applied science background with an undergraduate degree that was in some area that used statistics but did not involve deeper study of mathematics (e.g., economics, actuarial studies, finance, psychology, etc.). For these people, they usually hit a bit of a mathematical wall when starting postgraduate work in statistics, and they have to learn a lot of mathematical material that was absent in their undergraduate degree. This would usually include learning measure theory and set theory, and learning real analysis in greater depth than is usually covered in an applied science degree. Deeper learning of the foundations of mathematics would not usually be required, but it is not uncommon for people to dabble out of curiosity.

In terms of the depth of mathematical knowledge required for theoretical statistical work, for the most part, it is sufficent to have a solid understanding of logic and proofs (so that you can form theorems and prove them), and also have a good understanding of real analysis and measure theory. That is enough to understand the foundations of probability theory, which is where most statistical theory work starts. Knowledge of the foundations of mathematics is rarely required --- it is usually sufficient to take for granted that we can form a probability measure on a sigma-field of sets (e.g., the Borel sets) and start from there. Deeper foundational issues are left to logicians and mathematicians who work in that field.

Finally, it is worth noting that probabilists and statisticians often consider themselves somewhat similar to logicians, insofar as probability theory can be regarded as an "extension" of propositional (true-false) logic. This is particularly true for Bayesian statisticians, who often regard their work to be merely inductive logic, framed in mathematical form as an extension of propositional logic.

• I think it's germane that Harold Jeffreys called his main statistical book Theory of Probability and laid out axioms for the theory. The three editions were 1939, 1948, 1961. The word "Bayesian" was not in (widespread) use in 1939. His project fits your last sentence very well. He did include real data examples too, as did Harald Cramér, whose Mathematical Methods of Statistics (1946) went much further into mathematical foundations than was typical at the time. Aug 29 '20 at 8:43

Interesting question, is it possible maybe to find some references? See for instance Did Deborah Mayo refute Birnbaum's proof of the likelihood principle? a post about D Mayo's claimed refutation of the likelihood principle, a discussion where it seems some of the subtleties studied in mathematical logic enters.

Actually, changing search terms in google scholar to "formal logic" machine learning gives a lot more interesting-looking hits, which might just be hinting at something ...

Mathematical logic and axiomatic set theory are deeper, "lower" layers (or you can also call it "background"). You don't necessarily need to study them to be a good statistician. You will just apply them without even knowing it. But once you dive into them, you realize that they are actually keystones on which all the current science stands (well, math stands on them and current science stands on math). These disciplines formalize things which are considered so "obvious" and "natural" by scientists (i.e. axioms) that they don't even think about them.

I'd propose an analogy with computer world - if statistics would be an application, a program, then mathematical logic and set theory would symbolize the operating system. You can happily use the statistics without understanding the operating system.

One might imagine it like this: The blue boxes are actually these basic keystone layers which most people using applied statistics and mathematics won't need. The orange boxes are the applied disciplines which build on the below layers as if they were axioms. It is just an imperfect sketch of course, one might discuss if e.g. Mathematical analysis shouldn't be in the blue layers as well, and perpaphs aside from the Arithmetic... And also the orange boxes would perhaps form a network rather than independent boxes... so don't take it overly seriously :-) It just gives you an idea.