# What theories should every statistician know?

I'm thinking of this from a very basic, minimal requirements perspective. What are the key theories an industry (not academic) statistician should know, understand and utilize on a regular basis?

A big one that comes to mind is Law of large numbers. What are the most essential for applying statistical theory to data analysis?

## locked by whuber♦May 24 at 18:04

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. See the help center for guidance on writing a good question.

Frankly, I don't think the law of large numbers has a huge role in industry. It is helpful to understand the asymptotic justifications of the common procedures, such as maximum likelihood estimates and tests (including the omniimportant GLMs and logistic regression, in particular), the bootstrap, but these are distributional issues rather than probability of hitting a bad sample issues.

Beyond the topics already mentioned (GLM, inference, bootstrap), the most common statistical model is linear regression, so a thorough understanding of the linear model is a must. You may never run ANOVA in your industry life, but if you don't understand it, you should not be called a statistician.

There are different kinds of industries. In pharma, you cannot make a living without randomized trials and logistic regression. In survey statistics, you cannot make a living without Horvitz-Thompson estimator and non-response adjustments. In computer science related statistics, you cannot make a living without statistical learning and data mining. In public policy think tanks (and, increasingly, education statistics), you cannot make a living without causality and treatment effect estimators (which, increasingly, involve randomized trials). In marketing research, you need to have a mix of economics background with psychometric measurement theory (and you can learn neither of them in a typical statistics department offerings). Industrial statistics operates with its own peculiar six sigma paradigms which are but remotely connected to mainstream statistics; a stronger bond can be found in design of experiments material. Wall Street material would be financial econometrics, all the way up to stochastic calculus. These are VERY disparate skills, and the term "industry" is even more poorly defined than "academia". I don't think anybody can claim to know more than two or three of the above at the same time.

The top skills, however, that would be universally required in "industry" (whatever that may mean for you) would be time management, project management, and communication with less statistically-savvy clients. So if you want to prepare yourself for industry placement, take classes in business school on these topics.

UPDATE: The original post was written in February 2012; these days (March 2014), you probably should call yourself "a data scientist" rather than "a statistician" to find a hot job in industry... and better learn some Hadoop to follow with that self-proclamation.

• Great answer. Thank you for highlighting the some of the big differences between statisticians within industry. This helps motivate my question because I believe many people have a different idea of what a statistician is/does. I guess I was trying to find out where these all intersect from a basic understanding. Also, I really appreciate your last paragraph about business topics and how essential they are. Great points but I would still like to see if anyone can add to the conversation before accepting. – bnjmn Feb 17 '12 at 20:56
• I'm puzzled by these "peculiar Six Sigma paradigms", "remotely connected to mainstream Statistics" with which you say Industrial Statistics operates. It seems entirely orthodox to me, putting aside the differences in terminology found between all of these sub-fields. – Scortchi Mar 7 '14 at 16:02
• @Scortchi, I was unable to get past these terminology differences, frankly. I also know that normal approximations are close to being useless in the tails, so the 6 sigma probability $10^{-9}$ may be off by a factor of 100 or 1000. – StasK Mar 8 '14 at 17:22
• Fair enough: I'd have said measurement systems analysis (inter-rater agreement, gauge reproducibility & repeatability studies), statistical process control, reliability analysis (a.k.a. survival analysis), & experimental design ((fractional) factorial designs, response-surface methodology) were characteristic of industrial statistics. – Scortchi Mar 10 '14 at 21:25

I think a good understanding of the issues relating to the bias-variance tradeoff. Most statisticians will end up, at some point, analysing a dataset that is small enough for the variance of an estimator or the parameters of the model to be sufficiently high that bias is a secondary consideration.

To point out the super obvious one:

Central Limit Theorem

since it allows practitioners to approximate $p$-values in many situations where getting exact $p$-values is intractable. Along those same lines, any successful practitioner would be well served to be familiar, in general, with

Bootstrapping

I wouldn't say this is very similar to something like the law of large numbers or the central limit theorem, but because making inferences about causality is often central, understanding Judea Pearl's work on using structured graphs to model causality is something people should be familiar with. It provides a way to understand why experimental and observational studies differ with respect to the causal inferences they afford, and offers ways to deal with observational data. For a good overview, his book is here.

• There's also Rubin's counterfactuals framework; there are also structural equation modeling and econometric instrumental variable techniques... some of that described in the Mostly Harmless Econometrics which of the best statistics books written by non-statisticians. – StasK Mar 8 '14 at 17:17

A solid understanding of the substantive problem to be addressed is as important as any particular statistical approach. A good scientist in the industry is more likely than a statistician without such knowledge to come to a reasonable solution to their problem. A statistician with substantive knowledge can help.

The Delta-Method, how to calculate the variance of bizarre statistics and find their asymptotic relative efficiency, to recommend changes of variable and explain efficiency boosts by "estimating the right thing". In conjunction with that, Jensen's Inequality for understanding GLMs and strange kinds of bias which arise in transformations like above. And, now that bias and variance are mentioned, the concept of the bias-variance trade-off and MSE as an objective measure of predictive accuracy.

In my view, statistical inference is most important for a practitioner. Inference has two parts: 1) Estimation & 2) Hypothesis testing. Hypothesis testing is important one. Since in estimation mostly a unique procedure, maximum likelihood estimation, followed and it is available most statistical package(so there is no confusion).

Frequent practitioners questions are around significant testing of difference or causation analysis. Important hypothesis tests can be find in this link .

Knowing about Linear models, GLM or in general statistical modelling is required for causation interpretation. I assume future of data analysis include Bayesian inference.

Casual inference is must. And how to address it's fundamental problem, you can't go back in time and not give someone a treatment. Read articles about rubin, fisher the founder of modern statistics student.).... What to learn to address this problem, proper randomisation and how Law of large numbers says things are properly randomised, Hypothesis testing ,Potential outcomes (holds against hetroscastisty assumption and is great with missingness ), matching (great for missingness but potential outcomes is better because it's more generalised, I mean why learn a ton of complicated things when you can only learn one complicated thing ), Bootstrap ,Bayesian statistics of course( Bayesian regression, naïve Bayesian regression, Bayesian factors) , and Non papmetric alternatives.

Normally in practice just follow these general steps ,

Regarding a previous comment you should genrally first start with an ANOVA (random effects or fixed effects, and transform continuous types into bins) then use a regression (which if you transform and alter can sometimes be as good as a ANOVA but never beat it) to see which specific treatments are significant,( apposed to doing multiple t test and using some correction like Holm methid) use a regression.

In the cases where you have to predict things use bayasian regression.

Missingness at more than 5% use potential outcomes

Another branch of data analytics is supervised machine learning which must be mentioned