# What should be taught first: Probability or Statistics?

I have newly joined as a faculty member in a math dept. of a reputed institution. I will be teaching the course Probability and Statistics at the undergraduate level. The institution already has a syllabus for this course which I am not very much satisfied with. In that syllabus, statistics is covered first, also estimation part is missing. I always thought basics of probability should be taught before teaching statistics. Can someone give some opinion on this? Also a suggestion for the topics that should be covered in such a course is greatly appreciated.

• It seems to me that basic understanding (not necessarily of too mathy concepts) of probability is required to get the logic and intuition behind statistical techniques as evidenced by the fact that most elementary level stats books begin with probability. In my business undergrad class as well, we were first taught probability and then statistics. – rsl Jun 20 '16 at 8:08
• Is is theoretical, or applied statistics..? Much of the applied statistics can be thought without detailed introduction to probability. It could also be a nice apetizer for probability and theoretical statistics later on. So it doesn't have to be such bad idea. – Tim Jun 20 '16 at 8:16
• What degree program(s) are the students following ? Are they maths majors, or is the module offered mainly to students from other departments, say sociology or business ? – Robert Long Jun 20 '16 at 10:22
• @RobertLong: It's said that mechanical engg. students and our own math major students will be attending the course. Others from economics electrical engg. and computer science students might also attend. – Ashok Jun 20 '16 at 10:39

It doesn't seem to be a question of opinion any more: the world appears to have moved well beyond the traditional "teach probability and then teach statistics as an application of it." To get a sense of where the teaching of statistics is going, look at the list of paper titles in last year's special edition of The American Statistician (reproduced below): not a single one of them refers to probability.

They do discuss the teaching of probability and its role in the curriculum. A good example is George Cobb's paper and its responses. Here are some relevant quotations:

Modern statistical practice is much broader than is recognized by our traditional curricular emphasis on probability-­based inference.

What we teach lags decades behind what we practice. Our curricular paradigm emphasizes formal inference from a frequentist orientation, based either on the central limit theorem at the entry level or, in the course for mathematics majors, on a small set of parametric probability models that lend themselves to closed-­form solutions derived using calculus. The gap between our half-­century‐old curriculum and our contemporary statistical practice continues to widen.

My thesis ... is that as a profession we have only begun to explore the possibilities. The history of our subject also supports this thesis: Unlike probability, a scion of mathematics, statistics sprouted de novo from the soil of science.

Probability is a notoriously slippery concept. The gap between intuition and formal treatment may be wider than in any other branch of applied mathematics. If we insist that statistical thinking must necessarily be based on a probability model, how do we reconcile that requirement with goals of making central ideas “simple and approachable” and minimizing “prerequisites to research”?

As a thought experiment, run through the basic concepts and theory of estimation. Note how almost all of them can be explained and illustrated using only first-­semester calculus, with probability introduced along the way.

Of course we want students to learn calculus and probability, but it would be nice if we could join all the other sciences in teaching the fundamental concepts of our subject to first year students.

There's far more like this. You can read it yourself; the material is freely available.

### References

The special issue of the American Statistician on "Statistics and the Undergraduate Curriculum" (November, 2015) is available at http://amstat.tandfonline.com/toc/utas20/69/4.

Teaching the Next Generation of Statistics Students to “Think With Data”: Special Issue on Statistics and the Undergraduate Curriculum Nicholas J. Horton & Johanna S. Hardin DOI:10.1080/00031305.2015.1094283

Mere Renovation is Too Little Too Late: We Need to Rethink our Undergraduate Curriculum from the Ground Up George Cobb DOI:10.1080/00031305.2015.1093029

Teaching Statistics at Google-Scale Nicholas Chamandy, Omkar Muralidharan & Stefan Wager pages 283-291 DOI:10.1080/00031305.2015.1089790

Explorations in Statistics Research: An Approach to Expose Undergraduates to Authentic Data Analysis Deborah Nolan & Duncan Temple Lang DOI:10.1080/00031305.2015.1073624

Beyond Normal: Preparing Undergraduates for the Work Force in a Statistical Consulting Capstone Byran J. Smucker & A. John Bailer DOI:10.1080/00031305.2015.1077731

A Framework for Infusing Authentic Data Experiences Within Statistics Courses Scott D. Grimshaw DOI:10.1080/00031305.2015.1081106

Fostering Conceptual Understanding in Mathematical Statistics Jennifer L. Green & Erin E. Blankenship DOI:10.1080/00031305.2015.1069759

The Second Course in Statistics: Design and Analysis of Experiments? Natalie J. Blades, G. Bruce Schaalje & William F. Christensen DOI:10.1080/00031305.2015.1086437

A Data Science Course for Undergraduates: Thinking With Data Ben Baumer DOI:10.1080/00031305.2015.1081105

Data Science in Statistics Curricula: Preparing Students to “Think with Data” J. Hardin, R. Hoerl, Nicholas J. Horton, D. Nolan, B. Baumer, O. Hall-Holt, P. Murrell, R. Peng, P. Roback, D. Temple Lang & M. D. Ward DOI:10.1080/00031305.2015.1077729

Using Online Game-Based Simulations to Strengthen Students’ Understanding of Practical Statistical Issues in Real-World Data Analysis Shonda Kuiper & Rodney X. Sturdivant DOI:10.1080/00031305.2015.1075421

Combating Anti-Statistical Thinking Using Simulation-Based Methods Throughout the Undergraduate Curriculum Nathan Tintle, Beth Chance, George Cobb, Soma Roy, Todd Swanson & Jill VanderStoep DOI:10.1080/00031305.2015.1081619

What Teachers Should Know About the Bootstrap: Resampling in the Undergraduate Statistics Curriculum Tim C. Hesterberg DOI:10.1080/00031305.2015.1089789

Incorporating Statistical Consulting Case Studies in Introductory Time Series Courses Davit Khachatryan DOI:10.1080/00031305.2015.1026611

Developing a New Interdisciplinary Computational Analytics Undergraduate Program: A Qualitative-Quantitative-Qualitative Approach Scotland Leman, Leanna House & Andrew Hoegh DOI:10.1080/00031305.2015.1090337

From Curriculum Guidelines to Learning Outcomes: Assessment at the Program Level Beth Chance & Roxy Peck DOI:10.1080/00031305.2015.1077730

Program Assessment for an Undergraduate Statistics Major Allison Amanda Moore & Jennifer J. Kaplan DOI:10.1080/00031305.2015.1087331

• (+1) I think this is a great response. The problem with teaching probability theory first, is that it gives the impression that statistics is concerned with the specification, fitting and diagnostics of probabilistic models. Not only is this only a small part of statistics, but it skips over some of the most basic issues when thinking about data: how to visualize it and how to summarize it. What are the relative merits of mean or median to identify the location of a dataset? How should we cross-tabulate our results? What graphs are appropriate, and what are they telling us about the data? – Silverfish Jun 20 '16 at 23:33
• @Silverfish, it's so different from how guys like Bernoulli approached statistics. Take a look at this beautiful paper of his An attempt at a new analysis of the mortality caused by smallpox and of the advantages of inoculation to prevent it. He basically started modern epidemiology with it. If he were fixated on the things you mention like visualization etc., he wouldn't have brought the insight into the understanding of the underlying process. – Aksakal Jun 21 '16 at 12:59
• @Aksakal I'm not saying that modelling is unimportant, nor that a good dose of probability theory isn't vital. But statistics is wider than this. I think it's a mistake to view probability theory as foundational: or to the extent that it is, that this means it must be taught first. One may see irrational number like $\sqrt 2$ or $\pi$ for the first time in high school, but they don't introduce them via Dedekind cuts! Understanding data is something largely accessible to students without a math-priority background, and finding patterns is more important than p-value soup. – Silverfish Jun 21 '16 at 15:39
• For early breakthroughs in epidemiology, I'll raise you John Snow's study of the 1854 Broad Street cholera outbreak - something found in many good books introducing data visualization, and the thought processes for which must surely belong at the heart of Stats 101. – Silverfish Jun 21 '16 at 15:43
• @Aksakal I appreciate your mention of Bernoulli. However, how pioneers and great mathematicians approach and solve problems is altogether a different thing than how the solutions ought to be taught. I wish to emphasize that nobody (afaik) is advocating not teaching probability: the question concerns when such ideas should be introduced in the introductory (college) curriculum and to what extent they should be pursued. There are strong pedagogical reasons (grounded in psychology as well as teaching experience) for introducing many statistical ideas first. – whuber Jun 21 '16 at 15:58

The plural of anecdote isn't data, but in almost any course I've seen, at least the basics of probability comes before statistics.

On the other hand, historically, ordinary least squares was developed before the normal distribution was discovered! The statistical method came first, the more rigorous, probability based justification of why it works came second!

Stephen Stigler's History of Statistics: Measurement of Uncertainty Before 1900 takes the reader through the historical development:

• Mathematicians, astronomers understood basic mechanics and the law of gravity. They could describe the motion of heavenly bodies as a function of several parameters.
• They also had hundreds of observations of the celestial bodies, but how should the observations be combined to recover the parameters?
• A hundred observations gives you one hundred equations, but if there are only three unknowns to solve for, this is an overdetermined system...
• Legendre was first to develop the method of minimizing the sum of the square error. Later this was connected with the work in probability of Gauss and Laplace, that ordinary least squares was in some sense optimal given normally distributed errors.

## Why do I bring this up?

There's a certain logical elegance to first build up the mathematical machinery required to derive, understand some method, to lay the foundation before you build the house.

In the reality of science though, the house often comes first, the foundation second :P.

I'd love to see results from the education literature. What's more effective for teaching? What then why? Or why then what?

(I might be a weirdo, but I found the story of how least squares was developed to be an exciting page turner! Stories can make otherwise boring, abstract stuff come alive...)

• The statistical education literature appears to be consistent and unequivocal about this issue: probability is not statistics, statistics is far more than probability, and statistics (at least at the K-12 and college levels) should be introduced as thinking with data rather than as an application of probability. In addition to the references I provided in my answer, check out some other freely available resources such as CauseWeb, GAISE, and the J. Statistics Education. – whuber Jun 20 '16 at 16:22

I think it should be an iterative process for most people: you learn a little probability, then a little statistics, then a little more probability, and little more statistics etc.

For instance, take a look at the PhD Stat requirements at GWU. The PhD level Probability course 8257 has the following brief description:

STAT 8257. Probability. 3 Credits.
Probabilistic foundations of statistics, probability distributions, random variables, moments, characteristic functions, modes of convergence, limit theorems, probability bounds. Prerequisite: STAT 6201– STAT 6202, knowledge of calculus through functions of several variables and series.


Note, how it has Master's level statistics courses 6201 and 6202 in the pre-requisites. If you drill down to the lowest level stat or probability course in GWU, you'll get to Introduction to Business and Economic Statistics 1051 or Introduction to Statistics in Social Science 1053. Here's the description to one of them:

STAT 1051. Introduction to Business and Economic Statistics. 3 Credits.
Lecture (3 hours), laboratory (1 hour). Frequency distributions, descriptive measures, probability, probability distributions, sampling, estimation, tests of hypotheses, regression and correlation, with applications to business.


Notice, how the course has "Statistics" title but it teaches a probability within it. For many it's the first encounter with Probability theory after the high school "Stats" course.

This is somewhat similar to how it was taught in my days: the courses and textbooks were usually titled "Probability theory and mathematical statistics", e.g. Gmurman's text.

I can't imagine studying probability theory without any stats whatsoever. The PhD level course above 8257 assumes you already know statistics. So even if you first teach probability there will be some statistics learning involved. It's just for the first course it probably makes a sense to weigh a tad more on statistics, and use it to introduce probability theory too.

In the end it's an iterative process as I described in the beginning. And as in any good iterative process the first step is not important, whether the very first concept was from stats or probability won't matter after several iterations: you'll get to the same place regardless.

Final note, the teaching approach depends on your field. If you're studying physics, you'll get things like statistical mechanics, Fermi-Dirac statistics, which you're not going to deal with in social sciences. Also, in physics the frequentist approaches are natural, and in fact they're in the basis of some fundamental theories. Hence, it makes a sense to have a stand-alone probability theory taught early on, unlike social sciences where it may not make much sense to spend time on it and instead weigh more on statistics.

• A PhD curriculum, on the face of it, would seem to provide little guidance concerning how to teach a subject to undergraduates. The objectives and the audience are completely different. I will add a personal note: in my interactions with some established statistics departments, I have seen that the curriculum reflects 30-year-old ideas of pedagogy and progression and will continue to do so until a new generation of professors takes over. Perhaps we should be chary of using such curricula as normative models of what ought to be done. – whuber Jun 20 '16 at 16:27
• @whuber And then the curriculum will reflect then 30 year old ideas ... but at least they won't be 60 year old ideas. Anyhow, as it is said, science advances, funeral by funeral. – Mark L. Stone Jun 20 '16 at 16:47