I'm new to statistics and I have just come across the term "statistical confidence". I'm not too sure what it is or how it works but I would like to understand it first. I've tried to search for explanations online but they are all quite technical for me to understand. I would like to have a more intuitive explanation for a layman like me.

  • $\begingroup$ Welcome to Cross Validated! Please take a moment to view our tour. It is helpful for those who will be answering your question to know what you currently understand about the subject. $\endgroup$ – Tavrock Apr 9 '17 at 8:39
  • $\begingroup$ Is there a specific context in which you have come across the term? $\endgroup$ – Groovy_Worm Apr 9 '17 at 12:36
  • $\begingroup$ Have you searched our site for explanations? Please see stats.stackexchange.com/search?tab=votes&q=confidence%20. These results are sorted by votes so that you don't necessarily need to look at the many thousands of links they provide! $\endgroup$ – whuber Apr 9 '17 at 16:07

Start with the idea of probability. Probability represents the likelihood of random events that have not occurred yet. For example, if you are going to flip a coin then you can talk about the probability of it landing "Heads" as 50%.

But what if you have already flipped the coin, but have not looked at it yet. Since the random part is over the probability that it landed heads is either 0% or 100%, you just do not know which. We use the term "Confidence" to represent the combination of knowledge and uncertainty when the random part is over. We can say that we are 50% confident that the coin shows Heads.

Remember confidence is in the procedure.

Generally we talk about a confidence interval. For a specific interval we call it a 95% confidence interval and say that we are 95% confident that it contains the true value. We cannot use the term probability here because the random part is over, the specific interval either contains the true value (100% probability) or it does not (0% probability). But since we used a procedure that that gives intervals that contain the true parameter 95% of the time we say we have 95% confidence.

** Edit **

Based on @whuber's comments below I will expand a bit.

The linking of probability/confidence to time before and after randomness may be completely my own, I don't remember if I heard it somewhere or came up with it on my own. If someone else previously said this and I am just not remembering it, then I will happily give them credit, if not then I am willing to take the blame.

The more formal definitions of frequentist probability usually include a phrase more like "limit of the relative frequency of an event" and then go on to talk about probability in terms of general events or groups of possible outcomes. These definitions of probability deal with general concepts and ideas but do not apply to specific observed outcomes. I just flipped a coin and it is now sitting next to my keyboard. What is the "Probability" that this coin, at this time (until I move it) shows "Heads"? It is not 50%, even if we use the "Limit of relative frequency" definition. This specific case has a total number of trials equal to 1, so I have already reached the limit (it is not changing) and so the relative frequency of "Heads" is $\frac01$ or 0%.

Under the more formal definition of probability it is not very interesting (and non-intuitive) to talk about probability of a specific outcome, so we use the term "Confidence" instead to encompass the whole idea of "if I had flipped this coin many times, what frequency would I have seen heads" or other such phraseology. I just think that the before/after probability is a bit easier to understand, and the original poster asked for a lay explanation rather than a formal definition. This may be a case of Thumb's Second Postulate: "An easily understood, workable falsehood is more useful than a complex incomprehensible truth."

Another way to explain this is that probability is for the abstract, what might happen, what might have happened, what could happen if we did this again, etc. But probability becomes uninteresting in the concreate, specific example where there is only 1 observed outcome (so the relative frequency is either 0% or 100% whether we take a limit as n goes to 1 or not). But we can use "confidence" in the concrete, specific, observed case where we are still ignorant of some details.

My view here does not invalidate forensic or other applications, it may mean that the statements made would have been more correct if the word "confidence" was used instead of "probability" in some of the statements, but the same may be true under the more formal definition.

  • $\begingroup$ This strikes me as an unusual description of "confidence" and it appears to conflict with the sense of "confidence" used in confidence interval estimation or statements of confidence. In particular, it is difficult to distinguish from probability. Probability does not specifically apply to "when the random part is over"--at least, I know of no axiom of probability that either asserts or implies that. $\endgroup$ – whuber Nov 7 '18 at 20:43
  • $\begingroup$ @whuber, I am not sure that I fully understand your comment, but will try to expand. Your last sentence looks like you think I said that probability applies after the random portion when I said that probability only applies before the random part. To clarify, this is in frequentist statistics, Bayesian statistics uses probability after the fact as well, but has a whole different definition. My distinction of before/after random part is consistent with the first bullet point under "Misunderstandings" on the Wikipedia "Confidenc Intervals" page. $\endgroup$ – Greg Snow Nov 8 '18 at 16:39
  • $\begingroup$ Let me try another way: could you supply a reference to a reputable source that defines probability in terms of "events that have not occurred yet"? That characterization seems more appropriate for a discussion of stochastic processes, but does not appear to be essential for any theory of probability. Indeed, if we were to take you at your word, you seem to deny the applicability of frequentist methods to understanding past events. That would have a profound effect in the courtroom, the forensic sciences, and in many other areas of inquiry. $\endgroup$ – whuber Nov 8 '18 at 16:42
  • $\begingroup$ @whuber, the short answer is "no". I cannot (or at least will not at this time) provide the reference. The longer answer does not fit in a comment but has been added to the answer above. $\endgroup$ – Greg Snow Nov 8 '18 at 19:15

I'll take a stab at it. Say we are trying to estimate some parameter, $\theta$. Given some relevant data, we follow some prescribed recipe/formula and come up with an interval, $(\hat{\theta}_{low}, \hat{\theta}_{high})$. We are hopeful that we can have some confidence (in the layman's sense of the word) that the parameter is contained by this interval.

Statistical confidence is calculated based on some theoretical assumptions about the randomness in the data. If we follow this particular recipe and make some assumptions about the process that produced these data, the results of the recipe will give the desired result some percentage of the time.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy