There has been one question here regarding Taleb's article The Meta-Distribution of Standard P-Values. I have a somewhat different question, at the very end of the article Taleb claims that:

• One can safely see that under such stochasticity for the realizations of p-values and the distribution of its minimum, to get what people mean by 5% confidence (and the inferences they get from it), they need a p-value of at least one order of magnitude smaller.

On figure 2, there is a somewhat similar sounding claim, particularly the part "illusions of statistical significance":

Fig. 2. The probability distribution of a one-tailed p-value with expected value .11 generated by Monte Carlo (histogram) as well as analytically with ϕ(.) (the solid line). We draw all possible subsamples from an ensemble with given properties. The excessive skewness of the distribution makes the average value considerably higher than most observations, hence causing illusions of "statistical significance".

  1. What does Taleb mean by these statement?
  2. Is it correct that we should be using p-values order of magnitude lower to have "what people mean by 5% confidence"?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.