1
$\begingroup$

I often encounter (what I think is) incorrect reporting of confidence as 'probability'. This is possibly the single most common source of confusion regarding frequentist significance testing that a population statistic is seen as a random variable, not an unknown parameter.

However, I stumbled upon this passage from a very important and respectable source (https://climate.nasa.gov/evidence/):

The current warming trend is of particular significance because most of it is extremely likely (greater than 95 percent probability) to be the result of human activity since the mid-20th century and proceeding at a rate that is unprecedented over decades to millennia (1).

In the linked article, there are effect estimates that fall into 95% CIs and this is where (in my opinion) the above claim comes from. So, 95% of such constructed intervals would contain the true effect.

This got me thinking, whether I'm not overly strict about not mixing confidence and probability. When communicating the result to the general public, it seems to be common to speak in terms of likelihood, because people tend to think they have good grasp of it. Even if the studies show it is extremely subjective.

But my question is: do you think this simplification is justified? Or do I misunderstand the report they are linking to?

Update:

Assuming that the statement is indeed based on hypothesis testing, I would say that it is an example of a 'statistical colloquialism'. So to rephrase my question: as there is not one universal definition of probability, do you think it is acceptable to present the findings like this to the general public -- who has some intuitive understanding of the likelihood, based on the experience -- even if it not mathematically correct?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ It could be an innocent mistake, it could also refer to some estimate obtained not from hypothesis testing. $\endgroup$
    – user2974951
    Commented Sep 18, 2018 at 13:43
  • $\begingroup$ @user2974951 Sure, I think my question can be rephrased: how innocent is this mistake or whether I incorrectly assume that this comes from hypothesis testing :) $\endgroup$
    – Artur Nowak
    Commented Sep 18, 2018 at 13:46

1 Answer 1

Reset to default
4
$\begingroup$

The most formal way to state it is:

"Repeating the experiment an infinite number of times, and constructing the 95% CI in exactly the same way, the true population parameter lies inside the CI 95% of the time"

A common shorthand (and in my opinion correct) is:

"The true parameter lies inside the CI with 95% certainty/a probability of 95%"

with the understanding that the CI is random, not the population parameter.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Your first definition is the one I agree with (I included a shortened version of it in my question). I understand your reasoning why the shorthand is correct. However, this would only work if we approached infinite number of experiments, while we have just one experiment. To put it differently, it would mean that Bayesian credible intervals are the same as confidence intervals. I don't want to split hairs, but I seek a good way of communicating this. For me, 95% confidence was just that: not 100% confidence, but close to it. I don't try to express it as a probability. $\endgroup$
    – Artur Nowak
    Commented Sep 18, 2018 at 18:29
  • $\begingroup$ I found this answer that contains an example showing why "The true parameter lies inside the CI with 95% certainty/a probability of 95%" is not correct: stats.stackexchange.com/a/26457/11115. I updated the question to reflect my current understanding of the issue. $\endgroup$
    – Artur Nowak
    Commented Sep 19, 2018 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.