# Intuitive explanation of "Statistical Inference"

What is the cleanest, easiest way to explain someone the concept of Inference? What does it intuitively mean?

How would you go to explain it to the layperson, or to a person who has studied a very basic probability and statistics course?

something that would contribute to making it also 'intuitively' clear would be greatly appreciated!

• You ought to begin by making your meaning more precise, because for eons philosophers and scientists have recognized fundamentally different forms of inference, starting with the inductive/deductive split. Given the nature of this site, we ought to presuppose you mean some form of statistical inference, but there are many versions of those, too.
– whuber
Jul 2 '20 at 15:56

Sometimes it's best to explain a concept through a concrete example:

Imagine you grab an apple, take a bite from it and it tastes sweet. Will you conclude based on that bite that the entire apple is sweet? If yes, you will have inferred that the entire apple is sweet based on a single bite from it.

Inference is the process of using the part to learn about the whole.

How the part is selected is important in this process: the part needs to be representative of the whole. In other words, the part should be like a mini-me version of the whole. If it is not, our learning will be flawed and possibly incorrect.

Why do we need inference? Because we need to make conclusions and then decisions involving the whole based on partial information about it supplied by the part.

• Joined this community just to +1 this and say, not only is it by far the best answer here, it should be included in all textbooks and web explanations, as well. Marvelously well phrased. Jul 2 '20 at 21:59
• Thank you for the nicest comment I’ve ever received on this platform, Daniel, and welcome to the platform! The beauty of the CrossValidated is that you may get a wide range of answers to your question, covering different perspectives. I very much enjoyed reading the other answers in this fascinating thread and felt motivated to join it. So everyone who answered here deserves to share the warm glow coming from your lovely comment! ❤️ Jul 3 '20 at 2:25
• While this is certainly a beautiful answer (!), I'm not yet convinced that it really captures all of inference. Don't you sometimes (often?) also infer something about A from B, even when B is no part of A? Say, you perform a COVID-19 test and make inferences from the result about your health status. How is the test result a part of your health...? This part/whole formulation seems quite narrow to me.
– jhin
Jul 3 '20 at 12:47
• +1 to Daniel's comment, I really appreciate the simplicity of the answer, most often i see more senior users start using Mathematical Symbols and jargon, which makes it hard to digest for newcomers/beginners in the field. Kudos to @IsabellaGhement Jul 17 '20 at 18:07

Using the definition from All of Statistics by Larry A. Wasserman:

Statistical inference, or “learning” as it is called in computer science, is the process of using data to infer the distribution that generated the data. A typical statistical inference question is:

$$\textsf{Given a sample } X_1, \dots, X_n \sim F, \textsf{ how do we infer } F ?$$

In some cases, we may want to infer only some feature of $$F$$ such as its mean.

In statistics we interpret data as realizations of random variables, so what we learn in statistics are the characteristics of the random variables, i.e. things like distribution, expected value, variance, covariance, parameters of the distributions, etc. So statistical inference means learning those things from the data.

Citing E.T.Jaynes, "Probability theory: the logic of science" (a highly recommended read):

By 'inference' we mean simply: deductive reasoning whenever enough information is at hand to permit it; inductive or plausible reasoning when - as is almost invariably the case in real problems - the necessary information is not available. But if a problem can be solved by deductive reasoning, probability theory is not needed for it; thus our topic is the optimal processing of incomplete information.

In my own words, inference simply means to start from some given information and draw rational conclusions from it, where what's rational is usually defined by the rules of predicative logic or probability theory.

The information one uses for drawing conclusions may stem from beliefs one holds about the world (in technical jargon: models and prior distributions), from data that have been observed, or both. Of course, an inference can only be valid if the information it is based upon is valid!

If information is certain (you know things to be true or false), then the inference is performed by predicative logic: Aristotle is a man, men are no birds, therefore we infer that arostotle is no bird.

If information is uncertain (you believe things but are not certain), then the inference is performed by probability theory: if 50% of all people like pizza, and 50% of the people who like pizza also like pasta, while 75% of the people who don't' like pizza also don't like pasta, you can infer that - absent any further information - there is a 37.5% chance for you to like pasta. When you hear some kind of noise, based on your experiences you might be unsure whether the television or your little daughter is the source. You are drawing inferences - it's probably either the TV or your daughter - but you're unsure because the information provided is uncertain. When people talk about statistical inference, they usually refer to technical applications where one wants to use a lot of data to infer information about something that is not itself observable, just as in the last example.*

A typical technical example could go as follows: we have a temperature sensor in a room that returns a voltage $$V(k)$$. The sensor datasheet provides a graph that relates the measured voltage to temperature by a linear model: $$V(k) = a \cdot T(k) + b.$$ We may then use this model and the voltage measurements to draw inferences about the temperature in the room. Everything is deductive so far, because we assumed all information to be certain! Given $$V(k)$$, we can simply calculate $$T(k)$$.

We then observe that the estimated temperature fluctuates quite rapidly, much quicker than we would expect a room temperature to fluctuate. So we hypothesize that there is some kind of zero-mean, uncorrelated disturbance that also influences the sensor: $$V(k) = a \cdot T(k) + b + \epsilon(k).$$ We are now uncertain about the meaning of each voltage measurement (making each measurement an i.i.d. RV)! This tells us that we should average over a few voltage measurements to get a better estimate of the current room temperature.** If any of the information we used (the voltage-temperature model of the sensor, the disturbance model, the actual voltage measurements) is wrong, then our temperature estimate will also be wrong.

*Our brain is an extremely sophisticated inference device which draws all kinds of conclusions about ourselves, other people, our environment, and our future, all the time .

**Assuming that the sampling rate is much higher than the rate of change of the temperature and that the noise is really uncorrelated.

• Given where the question was asked, it is likely asking about statistical inference, not inference as in logic.
– Tim
Jul 2 '20 at 10:30
• There are huge differences between the two, starting with the stark difference between the certainty of logical inference ("deductive reasoning") and the uncertainty central to statistical inference.
– whuber
Jul 2 '20 at 15:54
• @whuber Well, please educate me if I'm missing something, but isn't logical inference a corollary of Bayesian inference? Assign probability 0 to false statements and probability 1 to true statements and perform standard Bayesian inference, and you have logical inference?
– jhin
Jul 2 '20 at 16:11
• Aside from the Jaynes text mentioned above, here is another paper which argues that both are two sides of the same coin. citeseerx.ist.psu.edu/viewdoc/…
– jhin
Jul 2 '20 at 16:12
• Of course, I'm not claiming that there is no difference at all between the two; that would be nonsense! All I wrote (and still believe) is that from a conceptual point of view, there is no difference: you take information, and you draw conclusions from that information by rational procedures. That is - to my mind - the core of "inference". If your information is certain, you get logical inference; if it is uncertain, you get statistical inference (both of which I discussed in my answer, by the way).
– jhin
Jul 2 '20 at 16:30

Statistical inference is the art of good guessing --- it entails guessing things that are unknown from related things that are known (observed), and giving associated measures of the level of confidence, variability, etc., in your guess.

• Nice definition, though I'm worried it might lack context for someone who doesn't understand what it is.
– Tim
Jul 2 '20 at 10:46
• Maybe. The goal of my little encapsulation of inference is to assist a layman who lacks any context at all. I actually think the beauty of this explanation is that it does not require contextual knowledge of distributions, samples, or any other statistical concept.
– Ben
Jul 2 '20 at 10:52

Let me try. The broad dictionary definition of inference is as follows:

something that you can find out indirectly from what you already know

And, from a more technical perspective, from The Oxford Dictionary of Statistical Terms by Upton, G., Cook I.,

statistical inference is the process of using data analysis to deduce properties of an underlying distribution of probability

Here, what we already know is the data (experiments we did) and sometimes a prior information. And, we want to know the properties of an entity of interest.

For example, say we have a biased coin and we want to have an idea on the probability of heads. We toss the coin a few times, record the results (which will be our data), and by looking at them, we'll have an understanding (which formally might be the distribution, moments etc.) of what the probability of heads looks like.

I'll try to rephrase Tim's answer since I think it's too technical for a layman.

Inference is the process of extracting (inferring) a general pattern from a particular set of cases. E.g., we have these particular data about soil, fertilizers and yield. What can we say about the general effect of soils and fertilizers on yield?

Probability, on the other hand, is somewhat the reverse exercise. We know the general pattern and we want to say something about particular cases. E.g., we know a die is fair. What can we say about the next 50 throws?

From the contents of two popular textbooks,

Casella and Berger (1990) -- Statistical Inference

Efron (2006) -- Computer Age Statistical Inference

I think statistical inference simply means mathematical and reasoning activities that try to make sense of data. More specifically, one may discern two approaches -- Bayesian and Frequentist, of which there are plenty of discussions on this site. I would point out that currently, most of the answers given to this question tend to have a Bayesian flavour. For example, trying to infer the underlying distribution of the data is a distinctly Bayesian activity. Frequentist inference is often more concerned with the procedure or algorithm that we apply to data, rather than the data itself. For example, one of the goals is to find the most powerful test of two hypothesis given the data. Judging by the contents of the book, it seems these activities also fall under the umbrella of statistical inference.

Lastly, I also need to point out that in the age of machine learning, the term inference has taken on a new meaning which is rather different from the above. In the training of neural networks, inference is simply the opposite of training. Whereas in training, a model is "built", in inference, the model is applied for prediction (typically in new data). See, for example, this article.

• In your final link. the word inferencing appears. If it means making inferences, i.e. inferring, then it is unnecessary; if it means something else then it is confusing. The real difficulty is that machine learning is actually concentrated on prediction, not inference, and its practitioners should acknowledge this distinction more. Jul 3 '20 at 12:47
• Inference could include prediction in the process, but prediction (of a test set) does not include inference. Jul 4 '20 at 3:09

Take this following case for example:
You want to know men's average height in the U.S. How could you proceed with this problem?
In an ideal situation, if you had unlimited time and energy, you can certainly collect the statistics from different resources and compiled them together to figure out the "undisputed truth" behind the scene, which, from statistician's point of view, often referred the population mean, or the expectation of a random variable $$X$$, denoted as $$E(X)$$, where $$X$$ represents men's height in the case.
Yet we are mortals with flesh so vulnerable to time, disease, and accidents, we only have limited time to do our job and find out the truth. The best thing we can do is to take a sample of our interest $$x_1,x_2,\ldots$$, then infer the truth from the imperfect mimic of undisputed truth $$E(X)$$. Besides that, the imperfect term has several interpretations:
1 The samples collected are prone to measurement errors, which may lead to a biased estimate of $$E(X)$$.
2 The samples surveyed might be not representative of entire population, which may drastically diverge from $$E(X)$$.
A very good analogy is to think of you sitting in front of a table, trying to figure out the contents in the Jigsaw puzzle. Suppose the number of pieces is infinite, of course you can't assemble each individual to fulfill your task, what the best can you do? If you picked up a bunch of pieces from central parts, you are very likely to get a rough estimate of the contents in a few attempts. What if you unfortunately picked the pieces from the corner sides? They are still the same shape, the same weight as the central pieces, but they are unrepresentative of the object in the picture. Beyond that, the central pieces collected is subject to your choice, which can sometimes lead to a biased estimate of "true" contents underlying in the picture.
In summary, statistical inference is the field of study that allows us to infer the undisputed truth from its representative part in a scientific, rigorous way.