Hot answers tagged

24 votes
Accepted

Why do we use hypothesis tests instead of just letting people do Bayesian updates?

Not a terrible idea, but you need to take account of decision-necessity, analysis-time-cost, and analysis-knowledge-requirements Hypothesis testing is useful in binary decision problems where one ...
Ben's user avatar
  • 118k
19 votes

Why do we use hypothesis tests instead of just letting people do Bayesian updates?

Welcome to CrossValidated. You’re right, a “bright line” of statistical “significance” has been proven to distort interpretations and to hurt science. Peer-reviewed journals and regulatory ...
Frank Harrell's user avatar
16 votes
Accepted

Can I use p-values without hypothesis?

You can't even calculate p values without a null hypothesis, because this is a crucial ingredient that goes into all tests, along with data and a model specification. For instance, for a t test for a ...
Stephan Kolassa's user avatar
15 votes

Why do we use hypothesis tests instead of just letting people do Bayesian updates?

Necessity is the mother of all invention. In the early 1900s there were no powerful computers which can run a Bayesian computation. Therefore, alternative methods need to be devised. Historically, ...
Nicolas Bourbaki's user avatar
14 votes

Why do we use hypothesis tests instead of just letting people do Bayesian updates?

I first say that I'm not against Bayesian analyses in general. However I believe that there is too little talk about some issues with Bayesian analyses that in many situations make them less ...
Christian Hennig's user avatar
12 votes

Can I use p-values without hypothesis?

Why do you even want a p value when you don't have a hypothesis? But, in short, the answer is "no". I mean, you can't stop the computer from outputting a p value, but, every p value has an ...
Peter Flom's user avatar
  • 110k
11 votes

Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons

My question is - is DSCF a post-hoc test which is supposed to only be applied when statistically significant differences are found on Kruskal-Wallis? Yes, the Kruskal-Wallis test is a non-parametric ...
Robert Long's user avatar
11 votes

$p$-value: Fisherian vs. contemporary frequentist definitions

My explanation of contemporary and past interpretations of $p$ values is more historical than mathematical. However, I list a number of references that discuss these things, both from the original ...
Shawn Hemelstrand's user avatar
11 votes

Are p-values computed from the a priori or a posteriori sampling distribution?

The null hypothesis is fixed before looking at the sample (or even planning the sample size). The p-value is obviously computed from the data, but it is based on the test statistic, which normally ...
Christian Hennig's user avatar
7 votes

Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons

What is ANOVA? It is important to note some of the reasons that ANOVA exist, as it can provide some context about how you should consider your omnibus ANOVA and the pairwise tests that accompany them (...
Shawn Hemelstrand's user avatar
6 votes

Why do we use hypothesis tests instead of just letting people do Bayesian updates?

Just to address the question in the title "Why do we use hypothesis tests instead of just letting people do bayesian updates?" directly... The reason is that Baysian probability was seen as ...
Dikran Marsupial's user avatar
6 votes

Are p-values computed from the a priori or a posteriori sampling distribution?

My question is: when we talk about the null hypothesis here, are we referring to the sampling distribution constructed: Before any data is collected (a priori), or After data has been collected (a ...
Dave's user avatar
  • 58.4k
5 votes

$p$-value: Fisherian vs. contemporary frequentist definitions

@ShawnHemelstrand's answer contains much of what is needed to get to an understanding of these issues, but there is something that I can add that will help, particularly with issues like those raised ...
Michael Lew's user avatar
  • 13.9k
5 votes

$p$-value: Fisherian vs. contemporary frequentist definitions

Extremity is defined in terms of the density of the test statistic First, $F$ tests and $\chi^2$ tests are typically rejected only for large values of the test statistic. Clearly, in Fisherian ...
Sextus Empiricus's user avatar
4 votes

Permutation test for exponential null hypothesis: really bad?

Edit: This answer doesn't deal with the actual meat of the question, which was about permutation tests. I will either fix it or delete it. The likelihood ratio test looks pretty good to me and ...
Glen_b's user avatar
  • 278k
4 votes

Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons

To me, this question is the same as whether to do any kind of pairwise comparisons after ANOVA. The fact that you used a nonparametric test doesn't change the key issues. This issue has been discussed ...
Peter Flom's user avatar
  • 110k
4 votes

The use of p-values in tests of d-separation

This is an excellent question. You're right in that testing for independence relies on some sort of significance threshold, and this is a challenge whenever one wants to discover properties of the DAG ...
Scriddie's user avatar
  • 1,762
4 votes

Test for synergy using Bliss independance model for two drugs

This could be quite a long answer as it's something I've also been working on recently. The Bliss Independence model (Bliss, 1939; Greco et al., 1995, Geary, 2013) is an Effect-Based method (in ...
Robert Long's user avatar
4 votes

$p$-value: Fisherian vs. contemporary frequentist definitions

Technically there is no contradiction. If you for example look at the one- or two-sided t-test for the $H_0$ of a mean being zero, the two-sided test will measure what's "more extreme" by ...
Christian Hennig's user avatar
3 votes

Are p-values computed from the a priori or a posteriori sampling distribution?

The p-value is computed with the sampling distribution Given a parameter $\theta \in \Theta$, an observed data vector $\mathbf{x}$ and a test statistic $T$ that is increasing with respect to evidence ...
Ben's user avatar
  • 118k
3 votes

A good fit of the model but the estimates are very low

It seems that the low numeric values of the coefficient estimates from your model are your primary concern. That's not a problem on its own, as the numeric values of those estimates depend on the ...
EdM's user avatar
  • 86.9k
2 votes

$p$-value: Fisherian vs. contemporary frequentist definitions

Shawn that is a wonderful answer but it’s important to point out a subtle problem with a small part of it. The “in the long run we should not often be wrong” statement of Neyman and Pearson is an ...
Frank Harrell's user avatar
2 votes

Do I always need to adjust the p value when using cor.test function in R?

There is a major difference between "I saw many p-values" and "I let this p-value determine how I should report my results". In fact, this has nothing to do with multiple testing. ...
AdamO's user avatar
  • 60.6k
2 votes

Which type of t-test is best to use for my data

In a comment you wrote N is the sample size for each year in the data. Rate is the percentage of that N that undertook an activity. This changes things a lot. Your sample size is not six (one for ...
Peter Flom's user avatar
  • 110k
1 vote

p-value comparison

No, you can't say more than whether one test was significant or not. Note that the difference between "significant" and "not significant” is not itself statistically significant" (...
Stephan Kolassa's user avatar
1 vote

Why do we use hypothesis tests instead of just letting people do Bayesian updates?

I agree with some other answers in that this could be a limitation of the toolset (the human mind), but, I think, perhaps not as simple a one as some people being too dumb to understand Bayesian ...
Jirka Hanika's user avatar
1 vote

Why do we use hypothesis tests instead of just letting people do Bayesian updates?

I don't think this is an issue of which method/algorithm/test we use, I think this is a much more fundamental issue of the people that we are using. Every approach has advantages and disadvantages, ...
user2974951's user avatar
  • 7,438
1 vote

Are p-values computed from the a priori or a posteriori sampling distribution?

I have caused confusion by using the terms "a priori" and "a posteriori". To acknowledge this I will use the phrases "before data collection" and "after data ...
Rez99's user avatar
  • 223
1 vote

Are p-values computed from the a priori or a posteriori sampling distribution?

The $p$-value is a posteriori because it depends on the observed data (or more precisely the observed test-statistic, $T^\star$), being $$ p = \text{Pr}(T \ge T^\star | H_0) $$ The type-1 and type-2 ...
innisfree's user avatar
  • 1,430
1 vote

High t statistic with high p value for same variable?

Your standard error is adjusted for only 2 clusters. That doesn't look right. It's hard to find a definitive answer to the minimum number of clusters (the book 'Mostly Harmless Econometrics' suggests ...
Jeremy Miles's user avatar
  • 16.5k

Only top scored, non community-wiki answers of a minimum length are eligible