Does learning thorough statistical theory require learning analysis?

Question

Does learning thorough statistical theory requires learning analysis before that? I looked at the textbook for statistical theory. So far I don't know if analysis is required, but I think I have heard analysis is a prerequisite. Should I learn analysis beforehand?

Answer 1

No, you do not need to know real analysis to learn statistics. In fact, in many universities (intro level) statistics courses are not even in the math department.

One can make a lot progress in statistics by letting the computer do all the math and you worrying only in how the statistical methods are being applied.

However, if you want to understand why the rules/tables are what they are then you need to know probability theory. The deeper you want to understand probability theory the more real analysis (really measure theory) you need to know. But at some point you reach diminishing returns. Sometimes you know too much and it just does not help you anymore in the uses of statistics.

So it is not required to know advanced math. However, knowing more (up to a certain extend without overdoing it) lets you apply it better and use better statistical techniques that you otherwise would not come up with.

Here are some books on statistics that do not use any measure theory: Bayesian Data Analysis by Gelman, Statistical Rethinking by McElreath, Doing Bayesian Data Analysis by Krushcke, Statistical Models by Freedman, Linear Models in Statistics by Rencher.

What do you notice about these books? These are all extremely well known books that are used to learn statistics, and they all avoid measure theory. In fact, there is not a single $\delta,\varepsilon$ proof in any one of them. McElreath's book even goes further to essentially eliminate all math from the subject and reduce it down to computer programming.

Now look at one of the most well-known and respected books on measure based probability theory, "Probability" by Shiryaev. There are virtually no statistical applications in that book anywhere. He does mention a few, but that is about it.

So if you are interested in learning statistics then you need to learn statistics. You can learn measure theory based probability on a need-to-know basis. However, if you start from the "ground up" and start with measure theory then you will hardly ever reach statistics in the end.

Answer 2

Yes, you do need a certain grounding in analysis and calculus to learn thorough statistical theory.

Note what I emphasized here. In order to apply statistics, e.g, fit models, interpret them, do null hypothesis significance testing, or create predictions, you don't need analysis (although it does help).

However, if you want to dig deeper - and that is what I take you to mean with "thorough statistical theory" - you will at the very least need to understand optimization, because all of model fitting involves optimizing something, such as maximizing a log-likelihood. And optimization is done through derivatives, typically higher ones. Similarly, even understanding expectations and higher moments of distributions involves infinite integrals and infinite sums, so you at least need to understand what symbols like "$\sum_{i=0}^\infty$" or "$\int_{-\infty}^\infty$" are supposed to mean. And this, you guessed it, is learned in analysis. Finally, if you want to go really deeply into statistical theory, as Nicolas says, you will need to learn measure theory, because all of probability in the end comes down to measure theory.

(For illustration, I personally am in the "applied" camp and sometimes venture into "theory land". I sometimes need a few derivatives, and regularly need to work with infinite sums or integrals. Measure theory never comes up in my personal work.)

Answer 3

Modern statistics is headed towards AI and ML. Not enough practitioners understand all the possible caveats and knowing measure theory, analysis and probability theory is the only way you can begin to understand the leading papers on the subject.

For example why is reproducibility a problem in lasso regression in high dimensional problems? The answer involves some fairly deep analysys.

Answer 4

Yet Bayesian methodology at least the subset of it you allude to, does not offer variable selection methods. The closest you can get is Bayesian selection which poses bernouli priors on variables in or out of the model and then say multivariate normal on the coefficients of variables in the model. The output of this is posterior probabilities for each variable being in or not and as such is not really selection at all. And no matter what methodology you settle on using, choice of models is always there. I suppose that your answer will be that the final model under the bayesian approach will be a mixture, for which inference is available. This may not be satisfactory for some situations.