What theories should every statistician know?

Question

I'm thinking of this from a very basic, minimal requirements perspective. What are the key theories an industry (not academic) statistician should know, understand and utilize on a regular basis?

A big one that comes to mind is Law of large numbers. What are the most essential for applying statistical theory to data analysis?

Answer 1

Frankly, I don't think the law of large numbers has a huge role in industry. It is helpful to understand the asymptotic justifications of the common procedures, such as maximum likelihood estimates and tests (including the omniimportant GLMs and logistic regression, in particular), the bootstrap, but these are distributional issues rather than probability of hitting a bad sample issues.

Beyond the topics already mentioned (GLM, inference, bootstrap), the most common statistical model is linear regression, so a thorough understanding of the linear model is a must. You may never run ANOVA in your industry life, but if you don't understand it, you should not be called a statistician.

There are different kinds of industries. In pharma, you cannot make a living without randomized trials and logistic regression. In survey statistics, you cannot make a living without Horvitz-Thompson estimator and non-response adjustments. In computer science related statistics, you cannot make a living without statistical learning and data mining. In public policy think tanks (and, increasingly, education statistics), you cannot make a living without causality and treatment effect estimators (which, increasingly, involve randomized trials). In marketing research, you need to have a mix of economics background with psychometric measurement theory (and you can learn neither of them in a typical statistics department offerings). Industrial statistics operates with its own peculiar six sigma paradigms which are but remotely connected to mainstream statistics; a stronger bond can be found in design of experiments material. Wall Street material would be financial econometrics, all the way up to stochastic calculus. These are VERY disparate skills, and the term "industry" is even more poorly defined than "academia". I don't think anybody can claim to know more than two or three of the above at the same time.

The top skills, however, that would be universally required in "industry" (whatever that may mean for you) would be time management, project management, and communication with less statistically-savvy clients. So if you want to prepare yourself for industry placement, take classes in business school on these topics.

Answer 2

To point out the super obvious one:

Central Limit Theorem

since it allows practitioners to approximate $p$-values in many situations where getting exact $p$-values is intractable. Along those same lines, any successful practitioner would be well served to be familiar, in general, with

Bootstrapping

Answer 3

I think a good understanding of the issues relating to the bias-variance tradeoff. Most statisticians will end up, at some point, analysing a dataset that is small enough for the variance of an estimator or the parameters of the model to be sufficiently high that bias is a secondary consideration.

Answer 4

I wouldn't say this is very similar to something like the law of large numbers or the central limit theorem, but because making inferences about causality is often central, understanding Judea Pearl's work on using structured graphs to model causality is something people should be familiar with. It provides a way to understand why experimental and observational studies differ with respect to the causal inferences they afford, and offers ways to deal with observational data. For a good overview, his book is here.

Answer 5

In my view, statistical inference is most important for a practitioner. Inference has two parts: 1) Estimation & 2) Hypothesis testing. Hypothesis testing is important one. Since in estimation mostly a unique procedure, maximum likelihood estimation, followed and it is available most statistical package(so there is no confusion).

Frequent practitioners questions are around significant testing of difference or causation analysis. Important hypothesis tests can be find in this link .

Knowing about Linear models, GLM or in general statistical modelling is required for causation interpretation. I assume future of data analysis include Bayesian inference.

Answer 6

The Delta-Method, how to calculate the variance of bizarre statistics and find their asymptotic relative efficiency, to recommend changes of variable and explain efficiency boosts by "estimating the right thing". In conjunction with that, Jensen's Inequality for understanding GLMs and strange kinds of bias which arise in transformations like above. And, now that bias and variance are mentioned, the concept of the bias-variance trade-off and MSE as an objective measure of predictive accuracy.

Answer 7

A solid understanding of the substantive problem to be addressed is as important as any particular statistical approach. A good scientist in the industry is more likely than a statistician without such knowledge to come to a reasonable solution to their problem. A statistician with substantive knowledge can help.