17
$\begingroup$

I've read through this thread, and it looks to me like it can be said that:

  • statistics = induction ?
  • probability = deduction ?

But I am wondering if there might be some more details on the comparison that I am missing. For example, is statistics equal to induction, or is it just a particular case of it? It does seem that probability is a sub case of deduction (since it is a sub case of mathematical thinking).

I know this is a picky question, but in a sense this is why I am asking it - because I want to be sure how these terms can be compared accurately.

$\endgroup$
15
$\begingroup$

I think it is the best to quickly recap the meaning of inductive and deductive reasoning before answering your question.

  • Deductive Reasoning: "Deductive arguments are attempts to show that a conclusion necessarily follows from a set of premises. A deductive argument is valid if the conclusion does follow necessarily from the premises, i.e., if the conclusion must be true provided that the premises are true. A deductive argument is sound if it is valid and its premises are true. Deductive arguments are valid or invalid, sound or unsound, but are never false or true." (quoted from wikipedia, emphasis added).

  • "Inductive reasoning, also known as induction or inductive logic, or educated guess in colloquial English, is a kind of reasoning that allows for the possibility that the conclusion is false even where all of the premises are true. The premises of an inductive logical argument indicate some degree of support (inductive probability) for the conclusion but do not entail it; that is, they do not ensure its truth." (from wikipedia, emphasis added)

To stress the main difference: Whereas deductive reasoning transfers the truth from premises to conclusions, inductive reasoning does not. That is, whereas for deductive reasoning you never broaden your knowledge (i.e., everything is in the premises, but sometimes hidden and needs to be demonstrated via proofs), inductive reasoning allows you to broaden your knowledge (i.e., you may gain new insights that are not already contained in the premises, however, for the cost of not knowing their truth).

How does this relate to probability and statistics?

In my eyes, probability is necessarily deductive. It is a branch of math. So based on some axioms or ideas (supposedly true ones) it deduces theories.

However, statistics is not necessarily inductive. Only if you try to use it for generating knowledge about unobserved entities (i.e., pursuing inferential statistics, see also onestop's answer). However, if you use statistics to describe the sample (i.e., decriptive statistics) or if you sampled the whole population, it is still deductive as you do not get any more knowledge or information as that is already present in the sample.

So, if you think about statistics as being the heroic endeavor of scientists trying to use mathematical methods to find regularities that govern the interplay of the empirical entities in the world, which is in fact never successful (i.e., we will never really know if any of our theories is true), then, yeah, this is induction. It's also the Scientific Method as articulated by Francis Bacon, upon which modern empirical science is founded. The method leads to inductive conclusions which are at best highly probable, though not certain. This in turn leads to misunderstanding among non-scientists about the meaning of a scientific theory and a scientific proof.


Update: After reading Conjugate Prior's answer (and after some overnight thinking) I would like to add something. I think the question on whether or not (inferential) statistical reasoning is deductive or inductive depends on what exactly it is that you are interested in, i.e., what kind of conclusion you are striving for.

If you are interested in probabilistic conclusions, then statistical reasoning is deductive. This means, if you want to know if e.g., in 95 out of 100 cases the population value is within a certain interval (i.e., confidence interval) , then you can get a truth value (true or not true) for this statement. You can say (if the assumptions are true) that it is the case that in 95 out of 100 cases the population value is within the interval. However, in no empirical case you will know if the population value is in your obtained CI. Either it is or not, but there is no way to be sure. The same reasoning applies for probabilities in classical p-value and Bayesian statistics. You can be sure about probabilities.

However, if you are interested in conclusions about empirical entities (e.g., where is the population value) you can only argue inductive. You can use all available statistical methods to accumulate evidence that support certain propositions about empirical entities or the causal mechanisms with which they interact. But you will never be certain on any of these propositions.

To recap: The point I want to make that it is important at what you are looking. Probabilites you can deduce, but for every definite proposition about things you can only find evidence in favor for. Not more. See also onestop's link to the induction problem.

$\endgroup$
  • $\begingroup$ Thank you Henrik - the distinction between the definitions (and your thoughts about them) was helpful. $\endgroup$ – Tal Galili Oct 29 '10 at 6:48
  • $\begingroup$ Your update was clear and to the point. If I could have given you another (+1), I would. $\endgroup$ – Tal Galili Oct 30 '10 at 14:44
7
$\begingroup$

Statistics is the deductive approach to induction. Consider the two main approaches to statistical inference: Frequentist and Bayesian.

Assume you are a Frequentist (in the style of Fisher, rather than Neyman for convenience). You wonder whether a parameter of substantive interest takes a particular value, so you construct a model, choose a statistic relating to the parameter, and perform a test. The p-value generated by your test indicates the probability of seeing a statistic as or more extreme than the statistic computed from the sample you have, assuming that your model is correct. You get a small enough p-value so you reject the hypothesis that the parameter does take that value. Your reasoning is deductive: Assuming the model is correct, either the parameter really does take the value of substantive interest but yours is an unlikely sample to see, or it does not take in fact that value.

Turning from hypothesis test to confidence intervals: you have a 95% confidence interval for your parameter which does not contain the value substantive interest. Your reasoning is again deductive: assuming the model is correct, either this is one of those rare intervals that will appear 1 in 20 times when the parameter really does have the value of substantive interest (because your sample is an unlikely one), or the parameter does not in fact have that value.

Now assume you are a Bayesian (in the style of Laplace rather than Gelman). Your model assumptions and calculations give you a (posterior) probability distribution over the parameter value. Most of the mass of this distribution is far from the value of substantive interest, so you conclude that the parameter probably does not have this value. Your reasoning is again deductive: assuming your model to be correct and if the prior distribution represented your beliefs about the parameter, then your beliefs about it in the light of the data are described by your posterior distribution which puts very little probability on that value. Since this distribution offers little support for the value of substantive interest, you might conclude that the parameter does not in fact have the value. (Or you might be content to state the probability it does).

In all three cases you get a logical disjunction to base your action on which is derived deductively/mathematically from assumptions. These assumptions are usually about a model of how the data is generated, but may also be prior beliefs about other quantities.

$\endgroup$
  • 1
    $\begingroup$ Thank you C.p, you make an interesting point. Although, from the perspective of Henrik's answer above, you are still in the realm of inductive, since the statistical reasoning you describe is one that involves uncertainty. $\endgroup$ – Tal Galili Oct 29 '10 at 6:51
  • $\begingroup$ Please see the (hopefully understandable) update to my answer, where I try to address the issue brought up here. $\endgroup$ – Henrik Oct 30 '10 at 14:17
  • $\begingroup$ @Henrik That is clearer (to me at least). Just a little niggle: It's not quite the case that "the same reasoning applies for probabilities in classical p-value and Bayesian statistics". The latter will give you single event probabilities, e.g. the probability that the true mean is between some value and some other value (although your other caveats all apply) while 'classical' frequentist methods such as confidence intervals will not even do that, despite the fond and widespread hope that they do. Their interpretation is indeed as you describe it. $\endgroup$ – conjugateprior Oct 30 '10 at 14:29
3
$\begingroup$

Yes! Maybe statistics isn't strictly equal to induction, but statistics is the solution to the problem of induction in my opinion.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.