Linked Questions
4
votes
4
answers
2k
views
How can I reduce a very large sample size for statistical significance (sampling methods)? [duplicate]
In my biological study, I have around 14000 independent samples, and I study the evolution of a response variable over time. I have three groups to study.
Thus, I have two factors: factor "Group&...
- 579
66
votes
14
answers
23k
views
If we fail to reject the null hypothesis in a large study, isn't it evidence for the null?
A basic limitation of null hypothesis significance testing is that it does not allow a researcher to gather evidence in favor of the null (Source)
I see this claim repeated in multiple places, but I ...
- 1,299
69
votes
8
answers
8k
views
What is a good, convincing example in which p-values are useful?
My question in the title is self explanatory, but I would like to give it some context.
The ASA released a statement earlier this week “on p-values: context, process, and purpose”, outlining various ...
- 21.6k
28
votes
8
answers
5k
views
Why are hypothesis tests still used when we have the bootstrap and central limit theorem?
Why are hypothesis tests still used when we have the bootstrap and central limit theorem?
To give context to my question, I briefly go over the central limit theorem and illustrate a simulation ...
19
votes
8
answers
3k
views
Interpreting non-statistically significant results: Do we have "no evidence" or "insufficient evidence" to reject the null?
When we fail to reject the null hypothesis in hypothesis testing, which of the below is the best interpretation?
We have no evidence at our significance level $\alpha$ to reject $H_0$.
We have ...
- 191
37
votes
5
answers
8k
views
Is p-value essentially useless and dangerous to use?
This article "The Odds, Continually Updated" from NY Times happened to catch my attention. To be short, it states that
[Bayesian statistics] is proving especially useful in approaching complex ...
- 2,292
15
votes
5
answers
8k
views
Are effect sizes really superior to p-values?
Lots of emphasis is placed on relying on and reporting effect sizes rather than p-values in applied research (e.g. quotes further below).
But is it not the case that an effect size just like a p-...
- 3,996
10
votes
6
answers
3k
views
How do you know if an insignificant P value is because of chance error or because the sample size of the data is too small?
I have always thought that an insignificant P value is the product of chance causing variation in the data. However when I was watching a talk on a study, the presenter mentioned how the insignificant ...
- 221
32
votes
5
answers
3k
views
How should an individual researcher think about the false discovery rate?
I've been trying to wrap my head around how the False Discovery Rate (FDR) should inform the conclusions of the individual researcher. For example, if your study is underpowered, should you discount ...
- 1,209
17
votes
4
answers
2k
views
Acceptance of null hypothesis
This is a discussion question on the intersection of statistics and other sciences. I often face the same problem: researchers in my field tend to say that there is no effect when the p-value is not ...
- 1,731
41
votes
2
answers
9k
views
Should we address multiple comparisons adjustments when using confidence intervals?
Suppose we have a multiple comparisons scenario such as post hoc inference on pairwise statistics, or like a multiple regression, where we are making a total of $m$ comparisons. Suppose also, that we ...
- 29.9k
9
votes
3
answers
6k
views
Understanding $\chi^{2}$ and Cramér's $V$ results
I have two groups of buyers, A and B, and I want to test whether the difference between the percentage of them who would buy a ...
- 537
7
votes
2
answers
865
views
Coefficient of 0.001 with p < 0.005 [duplicate]
This should be a simple inquiry. Doing a regression analysis I found that the coefficient of a predictor has a(n) (infinitesimal) positive effect of 0.001 that is significant at the 0.005 level.
I ...
- 103
11
votes
2
answers
825
views
In what settings would confidence intervals not get better as sample size increases?
In a blog post, I have found the claim that
"I believe WG Cochrane the first point out (roughly 1970′s) that with
confidence intervals in an observational setting, small sample sizes
result in ...
- 1,824
16
votes
2
answers
20k
views
Is it possible to accept the alternative hypothesis?
I'm aware of several related questions here (e.g., Hypothesis testing terminology surrounding null, Is it possible to prove a null hypothesis?) but I don't know the definitive answer for my question ...
- 161