Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
What is the cleanest, easiest way to explain someone the concept of Inference? What does it intuitively mean?
How would you go to explain it to the layperson, or to a person who has studied a very basic probability and statistics course?
something that would contribute to making it also 'intuitively' clear would be greatly appreciated!
Sometimes it's best to explain a concept through a concrete example:
Imagine you grab an apple, take a bite from it and it tastes sweet. Will you conclude based on that bite that the entire apple is sweet? If yes, you will have inferred that the entire apple is sweet based on a single bite from it.
Inference is the process of using the part to learn about the whole.
How the part is selected is important in this process: the part needs to be representative of the whole. In other words, the part should be like a mini-me version of the whole. If it is not, our learning will be flawed and possibly incorrect.
Why do we need inference? Because we need to make conclusions and then decisions involving the whole based on partial information about it supplied by the part.
I'm assuming that you're asking in here about statistical inference.
Using the definition from All of Statistics by Larry A. Wasserman:
Statistical inference, or “learning” as it is called in computer science, is the process of using data to infer the distribution that generated the data. A typical statistical inference question is:
$$ \textsf{Given a sample } X_1, \dots, X_n \sim F, \textsf{ how do we infer } F ? $$
In some cases, we may want to infer only some feature of $F$ such as its mean.
In statistics we interpret data as realizations of random variables, so what we learn in statistics are the characteristics of the random variables, i.e. things like distribution, expected value, variance, covariance, parameters of the distributions, etc. So statistical inference means learning those things from the data.
Citing E.T.Jaynes, "Probability theory: the logic of science" (a highly recommended read):
By 'inference' we mean simply: deductive reasoning whenever enough information is at hand to permit it; inductive or plausible reasoning when - as is almost invariably the case in real problems - the necessary information is not available. But if a problem can be solved by deductive reasoning, probability theory is not needed for it; thus our topic is the optimal processing of incomplete information.
In my own words, inference simply means to start from some given information and draw rational conclusions from it, where what's rational is usually defined by the rules of predicative logic or probability theory.
The information one uses for drawing conclusions may stem from beliefs one holds about the world (in technical jargon: models and prior distributions), from data that have been observed, or both. Of course, an inference can only be valid if the information it is based upon is valid!
If information is certain (you know things to be true or false), then the inference is performed by predicative logic: Aristotle is a man, men are no birds, therefore we infer that arostotle is no bird.
If information is uncertain (you believe things but are not certain), then the inference is performed by probability theory: if 50% of all people like pizza, and 50% of the people who like pizza also like pasta, while 75% of the people who don't' like pizza also don't like pasta, you can infer that - absent any further information - there is a 37.5% chance for you to like pasta. When you hear some kind of noise, based on your experiences you might be unsure whether the television or your little daughter is the source. You are drawing inferences - it's probably either the TV or your daughter - but you're unsure because the information provided is uncertain. When people talk about statistical inference, they usually refer to technical applications where one wants to use a lot of data to infer information about something that is not itself observable, just as in the last example.*
A typical technical example could go as follows: we have a temperature sensor in a room that returns a voltage $V(k)$. The sensor datasheet provides a graph that relates the measured voltage to temperature by a linear model: $$ V(k) = a \cdot T(k) + b.$$ We may then use this model and the voltage measurements to draw inferences about the temperature in the room. Everything is deductive so far, because we assumed all information to be certain! Given $V(k)$, we can simply calculate $T(k)$.
We then observe that the estimated temperature fluctuates quite rapidly, much quicker than we would expect a room temperature to fluctuate. So we hypothesize that there is some kind of zero-mean, uncorrelated disturbance that also influences the sensor: $$ V(k) = a \cdot T(k) + b + \epsilon(k).$$ We are now uncertain about the meaning of each voltage measurement (making each measurement an i.i.d. RV)! This tells us that we should average over a few voltage measurements to get a better estimate of the current room temperature.** If any of the information we used (the voltage-temperature model of the sensor, the disturbance model, the actual voltage measurements) is wrong, then our temperature estimate will also be wrong.
*Our brain is an extremely sophisticated inference device which draws all kinds of conclusions about ourselves, other people, our environment, and our future, all the time [1][2][3].
**Assuming that the sampling rate is much higher than the rate of change of the temperature and that the noise is really uncorrelated.
Statistical inference is the art of good guessing --- it entails guessing things that are unknown from related things that are known (observed), and giving associated measures of the level of confidence, variability, etc., in your guess.
Let me try. The broad dictionary definition of inference is as follows:
something that you can find out indirectly from what you already know
And, from a more technical perspective, from The Oxford Dictionary of Statistical Terms by Upton, G., Cook I.,
statistical inference is the process of using data analysis to deduce properties of an underlying distribution of probability
Here, what we already know is the data (experiments we did) and sometimes a prior information. And, we want to know the properties of an entity of interest.
For example, say we have a biased coin and we want to have an idea on the probability of heads. We toss the coin a few times, record the results (which will be our data), and by looking at them, we'll have an understanding (which formally might be the distribution, moments etc.) of what the probability of heads looks like.
I'll try to rephrase Tim's answer since I think it's too technical for a layman.
Inference is the process of extracting (inferring) a general pattern from a particular set of cases. E.g., we have these particular data about soil, fertilizers and yield. What can we say about the general effect of soils and fertilizers on yield?
Probability, on the other hand, is somewhat the reverse exercise. We know the general pattern and we want to say something about particular cases. E.g., we know a die is fair. What can we say about the next 50 throws?
From the contents of two popular textbooks,
Casella and Berger (1990) -- Statistical Inference
Efron (2006) -- Computer Age Statistical Inference
I think statistical inference simply means mathematical and reasoning activities that try to make sense of data. More specifically, one may discern two approaches -- Bayesian and Frequentist, of which there are plenty of discussions on this site. I would point out that currently, most of the answers given to this question tend to have a Bayesian flavour. For example, trying to infer the underlying distribution of the data is a distinctly Bayesian activity. Frequentist inference is often more concerned with the procedure or algorithm that we apply to data, rather than the data itself. For example, one of the goals is to find the most powerful test of two hypothesis given the data. Judging by the contents of the book, it seems these activities also fall under the umbrella of statistical inference.
Lastly, I also need to point out that in the age of machine learning, the term inference has taken on a new meaning which is rather different from the above. In the training of neural networks, inference is simply the opposite of training. Whereas in training, a model is "built", in inference, the model is applied for prediction (typically in new data). See, for example, this article.
Take this following case for example:
You want to know men's average height in the U.S. How could you proceed with this problem?
In an ideal situation, if you had unlimited time and energy, you can certainly collect the statistics from different resources and compiled them together to figure out the "undisputed truth" behind the scene, which, from statistician's point of view, often referred the population mean, or the expectation of a random variable $X$, denoted as $E(X)$, where $X$ represents men's height in the case.
Yet we are mortals with flesh so vulnerable to time, disease, and accidents, we only have limited time to do our job and find out the truth. The best thing we can do is to take a sample of our interest $x_1,x_2,\ldots$, then infer the truth from the imperfect mimic of undisputed truth $E(X)$. Besides that, the imperfect term has several interpretations:
1 The samples collected are prone to measurement errors, which may lead to a biased estimate of $E(X)$.
2 The samples surveyed might be not representative of entire population, which may drastically diverge from $E(X)$.
A very good analogy is to think of you sitting in front of a table, trying to figure out the contents in the Jigsaw puzzle. Suppose the number of pieces is infinite, of course you can't assemble each individual to fulfill your task, what the best can you do? If you picked up a bunch of pieces from central parts, you are very likely to get a rough estimate of the contents in a few attempts. What if you unfortunately picked the pieces from the corner sides? They are still the same shape, the same weight as the central pieces, but they are unrepresentative of the object in the picture. Beyond that, the central pieces collected is subject to your choice, which can sometimes lead to a biased estimate of "true" contents underlying in the picture.
In summary, statistical inference is the field of study that allows us to infer the undisputed truth from its representative part in a scientific, rigorous way.
