Stats is not maths?

Question

Is stats maths or not?

Given that it's all numbers, mostly taught by maths departments and you get maths credits for it, I wonder whether people just mean it half-jokingly when they say it, like saying it's a minor part of maths, or just applied maths.

I wonder if something like statistics, where you can't build everything on basic axioms can be considered maths. For example, the $p$-value, which is a concept that arose to make sense of data, but it's not a logical consequence of more basic principles.

Answer 1

Mathematics deals with idealized abstractions that (almost always) have absolute solutions, or the fact that no such solution exists can generally be described fully. It is the science of discovering complex but necessary consequences from simple axioms.

Statistics uses math, but it is not math. It's educated guesswork. It's gambling.

Statistics does not deal with idealized abstractions (although it does use some as tools), it deals with real world phenomena. Statistical tools often make simplifying assumptions to reduce the messy real world data to something that fits into the problem domain of a solved mathematical abstraction. This allows us to make educated guesses, but that's really all that statistics is: the art of making very well informed guesses.

Consider hypothesis testing with p-values. Let's say we are testing some hypothesis with significance $\alpha = 0.01$, and after gathering data we find a p-value of $0.001$. So we reject the null hypothesis in favor of an alternative hypothesis.

But what is this p-value really? What is the significance? Our test statistic was developed such that it conformed to a particular distribution, probably student's t. Under the null hypothesis, the percentile of our observed test statistic is the p-value. In other words, the p-value gives the probability that we would get a value as far from the expectation of the distribution (or farther) as the observed test statistic. The signficance level is a fairly arbitrary rule-of-thumb cutoff: setting it to $0.01$ is equivalent to saying, "it's acceptable if 1 in 100 repetitions of this experiment suggest that we reject the null, even if the null is in fact true."

The p-value gives us the probability that we observe the data at hand given that the null is true (or rather, getting a bit more technical, that we observe data under the null hypothesis that gives us at least as extreme a value of the tested statistic as that which we found). If we're going to reject the null, then we want this probability to be small, to approach zero. In our specific example, we found that the probability of observing the data we gathered if the null hypothesis were true was just $0.1\%$, so we rejected the null. This was an educated guess. We never really know for sure that the null hypothesis is false using these methods, we just develop a measurement for how strongly our evidence supports the alternative.

Did we use math to calculate the p-value? Sure. But math did not give us our conclusion. Based on the evidence, we formed an educated opinion, but it's still a gamble. We've found these tools to be extremely effective over the last 100 years, but the people of the future may wonder in horror at the fragility of our methods.

Answer 2

Tongue firmly in cheek:

Einstein apparently wrote

As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.

so statistics is the branch of maths that describes reality. ;o)

I'd say statistics is a branch of mathematics in the same way that logic is a branch of mathematics. It certainly includes an element of philosophy, but I don't think it is the only branch of mathematics where that is the case (see e.g. Morris Kline, "Mathematics - The Loss of Certainty", Oxford University Press, 1980).

Answer 3

Well if you say that "something like statistics, where you can't build everything on basic axioms" then you should probably read about Kolmogorov's axiomatic theory of probability. Kolmogorov defines probability in an abstract and axiomatic way as you can see in this pdf on page 42 or here at the bottom of page 1 and next pages.

Just to give you a flavour of his abstract definitions, he defines a random variable as a 'measurable' function as explained in a more 'intuitive' way here : If a random variable is a function, then how do we define a function of a random variable

With a very limited number of axioms and using results from (again maths) measure theory he can define concepts are random variables, distributions, conditional probability, ... in an abstract way and derive all well known results like the law of large numbers, ... from this set of axioms. I advise you to give it a try and you will be surprised about the mathematical beauty of it.

For an explanation on p-values I refer to: Misunderstanding a P-value?

Answer 4

I have no rigorous or philosophical basis for answering this, but I've heard the "stats is not math" complaint often from people, usually physics types. I think people want guarantees certainty from their math, and statistics (usually) offers only probabilistic conclusions with associated p values. Actually, this is exactly what I love about stats. We live in a fundamentally uncertain world, and we do the best we can to understand it. And we do a great job, all things considered.

Answer 5

Statistical tests, models, and inference tools are formulated in the language of mathematics, and statisticians have mathematically proven thick books of very important and interesting results about them. In many cases, the proofs provide compelling evidence that the statistical tools in question are reliable and/or powerful.

Statistics and its community may not be "pure" enough for mathematicians of a certain taste, but it is definitely invested in math extremely deeply, and theoretical statistics is just as much a branch of mathematics as theoretical physics or theoretical computer science.

Answer 6

Maybe its because I'm a plebe and haven't taken any advanced mathematical courses, but I don't see why statistics isn't mathematics. The arguments here and on a duplicate question seem to argue two primary points as to why statistics isn't mathematics^*.

1. It isn't exact/certain, and as such relies on assumptions.
2. It applies math to problems and anytime you apply math it is no longer math.

Isn't exact and uses assumptions

Assumptions/approximations are useful for lots of math.

The properties of a triangle that I learned about in grade school I believe are considered true math, even though they don't hold true in non-Elucidean geometry. So clearly an admission of the limits, or stated another way "assuming XYZ the following is valid", to a branch of math doesn't disqualify the branch from being "true" math.

Calculus I'm certain would be considered a pure form of math, but limits are the core tool we built it on. We can keep calculating up to the limit, just as we can keep making a sample size larger, but neither give increased insight past a certain threshold.

Once you apply math it isn't math

The obvious contradiction here is we use math to prove mathematical theorems, and no one argues that proving mathematical theorems isn't math.

The next statement might be that thing x isn't math if you use math to get a result. That doesn't make any sense either.

The statement I would agree with is that when you use the results of a calculation to make a decision then the decision isn't math. That doesn't mean that the analysis leading up to the decision isn't math.

I think when we use statistical analysis all the math performed is real math. It is only once we hand the results to someone for interpretation does statistics exit mathematics. As such statistics and statisticians are doing real mathematics and are real mathematicians. It is the interpretation done by the business and/or the translation of the results to the business by the statistician that isn't math.

From the comments:

whuber said:

If you were to replace "statistics" by "chemistry," "economics," "engineering," or any other field that employs mathematics (such as home economics), it appears none of your argument would change.

I think the key difference between "chemistry", "engineering", and "balancing my checkbook" is that those fields just use existing mathematical concepts. It is my understanding that statisticians like Guass expanded the body of mathematical concepts. I believe ^{(this might be blatantly wrong)} that in order to earn a PhD in statistics you have to contribute, in some way, to expanding the body of mathematical concepts. Chemistry/Engineering PhD candidates don't have that requirement to my knowledge.

The distinction that statistics contributes to the body of mathematical concepts is what sets it apart from the other fields that merely use mathematical concepts.

---

*: The notable exception is this answer that effectively states the boundaries are artificial due to various social reasons. I think that is the only true answer, but where is the fun in that? ;)

Answer 7

The "difference" relies on: Inductive reasoning vs. Deductive reasoning vs. Inference. For instance, no mathematical theorem can tell what distribution or prior you can use for your data/model.

By the way, Bayesian statistics is an axiomatised area.

Answer 8

This may be a very unpopular opinion, but given the history and formulation of concepts of statistics (and probability theory), I consider statistics to be a subbranch of physics.

Indeed, Gauss initially formalized the least squares regression model in astronomical predictions. The majority of contributions to statistics before Fisher were from Physicists (or highly applied mathematicians whose work would be called Physics by today's standards): Lyapunov, De Moivre, Gauss, and one or more of the Bernoullis.

The overarching principle is the characterization of errors and seeming randomness propagated from an infinite number of unmeasured sources of variation. As experiments became harder to control, experimental errors needed to be formally described and accounted for to calibrate the preponderance of experimental evidence against the proposed mathematical model. Later, as particle physics delved into quantum physics, formalizing particles as random distributions gave a much more concise language to describe the seemingly uncontrollable randomness with photons and electrons.

The properties of estimators such as their mean (center of mass) and standard deviation (second moment of deviations) are very intuitive to physicists. The majority of limit theorems can be loosely connected to Murphy's law, i.e. that the limiting normal distribution is maximum entropy.

So statistics is a subbranch of physics.