Linked Questions

59
votes
13answers
16k views

If we fail to reject the null hypothesis in a large study, isn't it evidence for the null?

A basic limitation of null hypothesis significance testing is that it does not allow a researcher to gather evidence in favor of the null (Source) I see this claim repeated in multiple places, but I ...
64
votes
8answers
8k views

What is a good, convincing example in which p-values are useful?

My question in the title is self explanatory, but I would like to give it some context. The ASA released a statement earlier this week “on p-values: context, process, and purpose”, outlining various ...
36
votes
5answers
6k views

Is p-value essentially useless and dangerous to use?

This article "The Odds, Continually Updated" from NY Times happened to catch my attention. To be short, it states that [Bayesian statistics] is proving especially useful in approaching complex ...
14
votes
5answers
5k views

Are effect sizes really superior to p-values?

Lots of emphasis is placed on relying on and reporting effect sizes rather than p-values in applied research (e.g. quotes further below). But is it not the case that an effect size just like a p-...
30
votes
5answers
2k views

How should an individual researcher think about the false discovery rate?

I've been trying to wrap my head around how the False Discovery Rate (FDR) should inform the conclusions of the individual researcher. For example, if your study is underpowered, should you discount ...
13
votes
4answers
2k views

Acceptance of null hypothesis

This is a discussion question on the intersection of statistics and other sciences. I often face the same problem: researchers in my field tend to say that there is no effect when the p-value is not ...
6
votes
3answers
4k views

Understanding $\chi^{2}$ and Cramér's $V$ results

I have two groups of buyers, A and B, and I want to test whether the difference between the percentage of them who would buy a ...
27
votes
2answers
4k views

Should we address multiple comparisons adjustments when using confidence intervals?

Suppose we have a multiple comparisons scenario such as post hoc inference on pairwise statistics, or like a multiple regression, where we are making a total of $m$ comparisons. Suppose also, that we ...
11
votes
2answers
412 views

In what settings would confidence intervals not get better as sample size increases?

In a blog post, I have found the claim that "I believe WG Cochrane the first point out (roughly 1970′s) that with confidence intervals in an observational setting, small sample sizes result in ...
10
votes
2answers
10k views

Is it possible to accept the alternative hypothesis?

I'm aware of several related questions here (e.g., Hypothesis testing terminology surrounding null, Is it possible to prove a null hypothesis?) but I don't know the definitive answer for my question ...
10
votes
2answers
412 views

Can we accept the null in noninferiority tests?

In a usual t-test of means, using the usual hypothesis testing methods, we either reject the null or fail to reject the null but we never accept the null. One reason for this is that if we got more ...
4
votes
3answers
2k views

Comparing the mean difference between data measured by different equipment, t-test suitable?

My problem is as follows: I have 2 measuring devices, both using different measuring techniques, each trying to quantify properties of objects passing their measurement area. The devices measure the ...
1
vote
3answers
1k views

Highly significant Pearson's chi-squared test (goodness of fit) when observed & expected are very close

I have two arrays that I would like to do a Pearson's chi-squared test (goodness of fit). I want to test whether or not there is a significant difference between the expected and observed results. <...
7
votes
1answer
702 views

Does failure to reject the null in Neyman-Pearson approach mean that one should “accept” it?

Based on Neyman-Pearson Hypothesis Testing Theory, will the failure to reject a null hypothesis imply that one should accept the null hypothesis? (It would be absolutely absurd to have this claim if ...
8
votes
1answer
1k views

Intuitive explanation of differences between TOST and UMP tests for equivalence

Hypothesis tests for equivalence differ from the more common hypothesis tests for difference. In tests for difference, the null hypothesis is some form of "separate quantities are the same", and ...

15 30 50 per page