24
votes
Accepted
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
Not a terrible idea, but you need to take account of decision-necessity, analysis-time-cost, and analysis-knowledge-requirements
Hypothesis testing is useful in binary decision problems where one ...
- 118k
19
votes
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
Welcome to CrossValidated.
You’re right, a “bright line” of statistical “significance” has been proven to distort interpretations and to hurt science. Peer-reviewed journals and regulatory ...
- 86.9k
16
votes
Accepted
Can I use p-values without hypothesis?
You can't even calculate p values without a null hypothesis, because this is a crucial ingredient that goes into all tests, along with data and a model specification.
For instance, for a t test for a ...
- 118k
15
votes
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
Necessity is the mother of all invention. In the early 1900s there were no powerful computers which can run a Bayesian computation. Therefore, alternative methods need to be devised. Historically, ...
- 2,186
14
votes
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
I first say that I'm not against Bayesian analyses in general. However I believe that there is too little talk about some issues with Bayesian analyses that in many situations make them less ...
- 21.2k
12
votes
Can I use p-values without hypothesis?
Why do you even want a p value when you don't have a hypothesis?
But, in short, the answer is "no". I mean, you can't stop the computer from outputting a p value, but, every p value has an ...
- 110k
11
votes
Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons
My question is - is DSCF a post-hoc test which is supposed to only be applied when statistically significant differences are found on Kruskal-Wallis?
Yes, the Kruskal-Wallis test is a non-parametric ...
- 57k
11
votes
$p$-value: Fisherian vs. contemporary frequentist definitions
My explanation of contemporary and past interpretations of $p$ values is more historical than mathematical. However, I list a number of references that discuss these things, both from the original ...
- 8,689
11
votes
Are p-values computed from the a priori or a posteriori sampling distribution?
The null hypothesis is fixed before looking at the sample (or even planning the sample size). The p-value is obviously computed from the data, but it is based on the test statistic, which normally ...
- 21.2k
7
votes
Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons
What is ANOVA?
It is important to note some of the reasons that ANOVA exist, as it can provide some context about how you should consider your omnibus ANOVA and the pairwise tests that accompany them (...
- 8,689
6
votes
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
Just to address the question in the title "Why do we use hypothesis tests instead of just letting people do bayesian updates?" directly...
The reason is that Baysian probability was seen as ...
- 53.5k
6
votes
Are p-values computed from the a priori or a posteriori sampling distribution?
My question is: when we talk about the null hypothesis here, are we referring to the sampling distribution constructed:
Before any data is collected (a priori), or
After data has been collected (a ...
- 58.4k
5
votes
$p$-value: Fisherian vs. contemporary frequentist definitions
@ShawnHemelstrand's answer contains much of what is needed to get to an understanding of these issues, but there is something that I can add that will help, particularly with issues like those raised ...
- 13.9k
5
votes
$p$-value: Fisherian vs. contemporary frequentist definitions
Extremity is defined in terms of the density of the test statistic
First, $F$ tests and $\chi^2$ tests are typically rejected only for large values of the test statistic. Clearly, in Fisherian ...
- 71.9k
4
votes
Permutation test for exponential null hypothesis: really bad?
Edit: This answer doesn't deal with the actual meat of the question, which was about permutation tests. I will either fix it or delete it.
The likelihood ratio test looks pretty good to me and ...
- 278k
4
votes
Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons
To me, this question is the same as whether to do any kind of pairwise comparisons after ANOVA. The fact that you used a nonparametric test doesn't change the key issues.
This issue has been discussed ...
- 110k
4
votes
The use of p-values in tests of d-separation
This is an excellent question.
You're right in that testing for independence relies on some sort of significance threshold, and this is a challenge whenever one wants to discover properties of the DAG ...
- 1,762
4
votes
Test for synergy using Bliss independance model for two drugs
This could be quite a long answer as it's something I've also been working on recently. The Bliss Independence model (Bliss, 1939; Greco et al., 1995, Geary, 2013) is an Effect-Based method (in ...
- 57k
4
votes
$p$-value: Fisherian vs. contemporary frequentist definitions
Technically there is no contradiction. If you for example look at the one- or two-sided t-test for the $H_0$ of a mean being zero, the two-sided test will measure what's "more extreme" by ...
- 21.2k
3
votes
Are p-values computed from the a priori or a posteriori sampling distribution?
The p-value is computed with the sampling distribution
Given a parameter $\theta \in \Theta$, an observed data vector $\mathbf{x}$ and a test statistic $T$ that is increasing with respect to evidence ...
- 118k
3
votes
A good fit of the model but the estimates are very low
It seems that the low numeric values of the coefficient estimates from your model are your primary concern. That's not a problem on its own, as the numeric values of those estimates depend on the ...
- 86.9k
2
votes
$p$-value: Fisherian vs. contemporary frequentist definitions
Shawn that is a wonderful answer but it’s important to point out a subtle problem with a small part of it. The “in the long run we should not often be wrong” statement of Neyman and Pearson is an ...
- 86.9k
2
votes
Do I always need to adjust the p value when using cor.test function in R?
There is a major difference between "I saw many p-values" and "I let this p-value determine how I should report my results". In fact, this has nothing to do with multiple testing.
...
- 60.6k
2
votes
Which type of t-test is best to use for my data
In a comment you wrote
N is the sample size for each year in the data. Rate is the percentage of that N that undertook an activity.
This changes things a lot. Your sample size is not six (one for ...
- 110k
1
vote
p-value comparison
No, you can't say more than whether one test was significant or not. Note that the difference between "significant" and "not significant” is not itself statistically significant" (...
- 118k
1
vote
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
I agree with some other answers in that this could be a limitation of the toolset (the human mind), but, I think, perhaps not as simple a one as some people being too dumb to understand Bayesian ...
- 111
1
vote
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
I don't think this is an issue of which method/algorithm/test we use, I think this is a much more fundamental issue of the people that we are using. Every approach has advantages and disadvantages, ...
- 7,438
1
vote
Are p-values computed from the a priori or a posteriori sampling distribution?
I have caused confusion by using the terms "a priori" and "a posteriori". To acknowledge this I will use the phrases "before data collection" and "after data ...
- 223
1
vote
Are p-values computed from the a priori or a posteriori sampling distribution?
The $p$-value is a posteriori because it depends on the observed
data (or more precisely the observed test-statistic, $T^\star$),
being $$ p = \text{Pr}(T \ge T^\star | H_0) $$
The type-1 and type-2 ...
- 1,430
1
vote
High t statistic with high p value for same variable?
Your standard error is adjusted for only 2 clusters. That doesn't look right. It's hard to find a definitive answer to the minimum number of clusters (the book 'Mostly Harmless Econometrics' suggests ...
- 16.5k
Only top scored, non community-wiki answers of a minimum length are eligible