9
$\begingroup$

Does learning thorough statistical theory requires learning analysis before that? I looked at the textbook for statistical theory. So far I don't know if analysis is required, but I think I have heard analysis is a prerequisite. Should I learn analysis beforehand?

$\endgroup$
1
  • 10
    $\begingroup$ IMO you do not need to study analysis to apply statistics. But to 'thoroughly' understand the current state of stats I don't see that being possible without at least some analysis. But I don't think anyone thoroughly understands stats anyways so let me know if you figure out a way to do that. $\endgroup$
    – Galen
    Nov 19, 2023 at 5:43

4 Answers 4

15
$\begingroup$

No, you do not need to know real analysis to learn statistics. In fact, in many universities (intro level) statistics courses are not even in the math department.

One can make a lot progress in statistics by letting the computer do all the math and you worrying only in how the statistical methods are being applied.

However, if you want to understand why the rules/tables are what they are then you need to know probability theory. The deeper you want to understand probability theory the more real analysis (really measure theory) you need to know. But at some point you reach diminishing returns. Sometimes you know too much and it just does not help you anymore in the uses of statistics.

So it is not required to know advanced math. However, knowing more (up to a certain extend without overdoing it) lets you apply it better and use better statistical techniques that you otherwise would not come up with.

Here are some books on statistics that do not use any measure theory: Bayesian Data Analysis by Gelman, Statistical Rethinking by McElreath, Doing Bayesian Data Analysis by Krushcke, Statistical Models by Freedman, Linear Models in Statistics by Rencher.

What do you notice about these books? These are all extremely well known books that are used to learn statistics, and they all avoid measure theory. In fact, there is not a single $\delta,\varepsilon$ proof in any one of them. McElreath's book even goes further to essentially eliminate all math from the subject and reduce it down to computer programming.

Now look at one of the most well-known and respected books on measure based probability theory, "Probability" by Shiryaev. There are virtually no statistical applications in that book anywhere. He does mention a few, but that is about it.

So if you are interested in learning statistics then you need to learn statistics. You can learn measure theory based probability on a need-to-know basis. However, if you start from the "ground up" and start with measure theory then you will hardly ever reach statistics in the end.

$\endgroup$
2
  • 7
    $\begingroup$ Beautifully stated. Note that the books needing little advanced theory are mainly Bayesian books. Bayes cuts through so many things; it doesn't need sufficient statistics, ancillarity, large sample theory/asymptotics, delta method, approximating sampling distributions to get accurate p-values and confidence limits, etc. $\endgroup$ Nov 19, 2023 at 16:31
  • $\begingroup$ Thank you for the great answer! $\endgroup$
    – 070701
    Nov 24, 2023 at 2:26
12
$\begingroup$

Yes, you do need a certain grounding in analysis and calculus to learn thorough statistical theory.

Note what I emphasized here. In order to apply statistics, e.g, fit models, interpret them, do null hypothesis significance testing, or create predictions, you don't need analysis (although it does help).

However, if you want to dig deeper - and that is what I take you to mean with "thorough statistical theory" - you will at the very least need to understand optimization, because all of model fitting involves optimizing something, such as maximizing a log-likelihood. And optimization is done through derivatives, typically higher ones. Similarly, even understanding expectations and higher moments of distributions involves infinite integrals and infinite sums, so you at least need to understand what symbols like "$\sum_{i=0}^\infty$" or "$\int_{-\infty}^\infty$" are supposed to mean. And this, you guessed it, is learned in analysis. Finally, if you want to go really deeply into statistical theory, as Nicolas says, you will need to learn measure theory, because all of probability in the end comes down to measure theory.

(For illustration, I personally am in the "applied" camp and sometimes venture into "theory land". I sometimes need a few derivatives, and regularly need to work with infinite sums or integrals. Measure theory never comes up in my personal work.)

$\endgroup$
1
  • $\begingroup$ Thank you for the great answer! $\endgroup$
    – 070701
    Nov 24, 2023 at 2:26
3
$\begingroup$

Modern statistics is headed towards AI and ML. Not enough practitioners understand all the possible caveats and knowing measure theory, analysis and probability theory is the only way you can begin to understand the leading papers on the subject.

For example why is reproducibility a problem in lasso regression in high dimensional problems? The answer involves some fairly deep analysys.

$\endgroup$
3
  • 3
    $\begingroup$ I think many AI/ML researchers are happy to use methods that aren't accompanied by rigorous proofs of the requisite large sample properties, provided that the method performs well in benchmark tests. I think the theorists are trying to catch up with practitioners but the breakneck pace at which research moves in this area seems to suggest that they will be lagging behind for a while. $\endgroup$ Nov 19, 2023 at 16:18
  • 4
    $\begingroup$ Modern statistics is not necessarily headed towards ML. Bayesian modeling can handle amazing complexities (including shrinkage priors that work better than the penalty/regularization functions used in ML, lasso, elastic net) and provide exact inference. $\endgroup$ Nov 19, 2023 at 16:33
  • $\begingroup$ Thank you for the great answer! $\endgroup$
    – 070701
    Nov 24, 2023 at 2:26
1
$\begingroup$

Yet Bayesian methodology at least the subset of it you allude to, does not offer variable selection methods. The closest you can get is Bayesian selection which poses bernouli priors on variables in or out of the model and then say multivariate normal on the coefficients of variables in the model. The output of this is posterior probabilities for each variable being in or not and as such is not really selection at all. And no matter what methodology you settle on using, choice of models is always there. I suppose that your answer will be that the final model under the bayesian approach will be a mixture, for which inference is available. This may not be satisfactory for some situations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.