30 questions linked to/from Misunderstanding a P-value?
463k views

What is the meaning of p values and t values in statistical tests?

After taking a statistics course and then trying to help fellow students, I noticed one subject that inspires much head-desk banging is interpreting the results of statistical hypothesis tests. It ...
14k views

Is this really how p-values work? Can a million research papers per year be based on pure randomness?

I'm very new to statistics, and I'm just learning to understand the basics, including $p$-values. But there is a huge question mark in my mind right now, and I kind of hope my understanding is wrong. ...
7k views

Interpretation of p-value in hypothesis testing

I recently came across the paper "The Insignificance of Null Hypothesis Significance Testing", Jeff Gill (1999). The author raised a few common misconceptions regarding hypothesis testing and p-values,...
9k views

Stats is not maths?

Is stats maths or not? Given that it's all numbers, mostly taught by maths departments and you get maths credits for it, I wonder whether people just mean it half-jokingly when they say it, like ...
17k views

Comparing and contrasting, p-values, significance levels and type I error

I was wondering if anybody could give a concise rundown as to the definitions and uses of p-values, significance level and type I error. I understand that p-values are defined as "the probability of ...
9k views

Are smaller p-values more convincing?

I've been reading up on $p$-values, type 1 error rates, significance levels, power calculations, effect sizes and the Fisher vs Neyman-Pearson debate. This has left me feeling a bit overwhelmed. I ...
21k views

Is Benjamini-Hochberg correction more conservative as the number of comparisons increases?

How conservative is Benjamini-Hochberg multiple testing correction relative to the total number of comparisons? For example, if I have a list of 18,000 features for two groups and I perform a Wilcoxon ...
4k views

What follows if we fail to reject the null hypothesis? [duplicate]

What conclusions can we draw if $p>\alpha$? Does not rejecting the $H_0$ mean anything?
2k views

Using p-value to compute the probability of hypothesis being true; what else is needed?

Question: One common misunderstanding of p-values is that they represent the probability of the null hypothesis being true. I know that's not correct and I know that p-values only represent the ...
4k views

Probability that Null Hypothesis is True

So, this may be a common question, but I’ve never found a satisfactory answer. How do you determine the probability that the null hypothesis is true (or false)? Let’s say you give students two ...
2k views

What is the relationship between $p$ values and Type I errors [duplicate]

In hypothesis testing we set an accepted level of Type I error probability $\alpha$ and observe whether a sample statistic is equally likely or less likely to be observed if the null hypothesis was ...
15k views

Can p-value be greater than 1?

Say I perform Student's t-test: can the p-value be greater than 1? The reason I ask this is that, some say that the p-value is the probability that we will have a Type I error and I would like to ...
3k views

96 views

Why do we do hypothesis testing on estimates of linear regression?

I was reading about linear regression and what I understood is that once we minimize OLS equation we get the beta parameters. Its just like solving a normal equation to get the unknowns. Then why do ...
415 views

Confusion regarding p-values and false discovery rate

I get stuck with understanding of the following two statements (from Wikipedia on p-values): The p-value is the probability of obtaining at least as extreme results given that the null hypothesis ...
235 views

Is the p-postulate (equal p-values provide equal evidence against the null) true?

The p-postulate is the notion that equal p-values provide equal evidence against the null hypothesis. Wagenmakers et al (2008) write: If p-values truly reflect evidence, a minimum requirement is ...
79 views

Am I taking crazy pills here or this A/B testing tool's documentation dead wrong?

In the documentation here for a popular A/B testing they say, For example, if you run an A/B testing experiment with a significance level of 95%, this means that if you determine a winner, you ...
153 views

Is my interpretation of the p-value and confidence interval correct?

Background: let's say I have a website with a big buy button on it and I preformed an A/B testing with the following result: Version A (control): conversion rate 5% Version B (variation): ...