Citing E.T.Jaynes, "Probability theory: the logic of science" (a highly recommended read):
By 'inference' we mean simply: deductive reasoning whenever enough information is at hand to permit it; inductive or plausible reasoning when - as is almost invariably the case in real problems - the necessary information is not available. But if a problem can be solved by deductive reasoning, probability theory is not needed for it; thus our topic is the optimal processing of incomplete information.
In my own words, inference simply means to start from some given information and draw rational conclusions from it, where what's rational is usually defined by the rules of predicative logic or probability theory.
The information one uses for drawing conclusions may stem from beliefs one holds about the world (in technical jargon: models and prior distributions), from data that have been observed, or both. Of course, an inference can only be valid if the information it is based upon is valid!
If information is certain (you know things to be true or false), then the inference is performed by predicative logic: Aristotle is a man, men are no birds, therefore we infer that arostotle is no bird.
If information is uncertain (you believe things but are not certain), then the inference is performed by probability theory: if 50% of all people like pizza, and 50% of the people who like pizza also like pasta, while 75% of the people who don't' like pizza also don't like pasta, you can infer that - absent any further information - there is a 37.5% chance for you to like pasta. When you hear some kind of noise, based on your experiences you might be unsure whether the television or your little daughter is the source. You are drawing inferences - it's probably either the TV or your daughter - but you're unsure because the information provided is uncertain. When people talk about statistical inference, they usually refer to technical applications where one wants to use a lot of data to infer information about something that is not itself observable, just as in the last example.*
A typical technical example could go as follows: we have a temperature sensor in a room that returns a voltage $V(k)$. The sensor datasheet provides a graph that relates the measured voltage to temperature by a linear model:
$$ V(k) = a \cdot T(k) + b.$$
We may then use this model and the voltage measurements to draw inferences about the temperature in the room. Everything is deductive so far, because we assumed all information to be certain! Given $V(k)$, we can simply calculate $T(k)$.
We then observe that the estimated temperature fluctuates quite rapidly, much quicker than we would expect a room temperature to fluctuate. So we hypothesize that there is some kind of zero-mean, uncorrelated disturbance that also influences the sensor:
$$ V(k) = a \cdot T(k) + b + \epsilon(k).$$
We are now uncertain about the meaning of each voltage measurement (making each measurement an i.i.d. RV)! This tells us that we should average over a few voltage measurements to get a better estimate of the current room temperature.** If any of the information we used (the voltage-temperature model of the sensor, the disturbance model, the actual voltage measurements) is wrong, then our temperature estimate will also be wrong.
*Our brain is an extremely sophisticated inference device which draws all kinds of conclusions about ourselves, other people, our environment, and our future, all the time [1][2][3].
**Assuming that the sampling rate is much higher than the rate of change of the temperature and that the noise is really uncorrelated.