Linked Questions

60
votes
13answers
16k views

If we fail to reject the null hypothesis in a large study, isn't it evidence for the null?

A basic limitation of null hypothesis significance testing is that it does not allow a researcher to gather evidence in favor of the null (Source) I see this claim repeated in multiple places, but I ...
35
votes
7answers
6k views

Why is the null hypothesis often sought to be rejected?

I hope I am making sense with the title. Often, the null hypothesis is formed with the intention of rejecting it. Is there a reason for this, or is it just a convention?
16
votes
4answers
6k views

Misunderstanding a P-value?

So I've been reading a lot about how to correctly interpret a P-value, and from what I've read, the p-value says NOTHING about the probability that the null hypothesis is true or false. However, when ...
12
votes
1answer
5k views

Goodness-of-fit test in Logistic regression; which 'fit' do we want to test?

I am referring to the question and its answers: How to compare (probability) predictive ability of models developed from logistic regression? by @Clark Chong and answers/comments by @Frank Harrell. ...
11
votes
2answers
1k views

Family-wise error boundary: Does re-using data sets on different studies of independent questions lead to multiple testing problems?

If a team of researchers perform multiple (hypothesis) tests on a given data set, there is a volume of literature asserting that they should use some form of correction for multiple testing (...
11
votes
1answer
959 views

What are the ''desirable'' statistical properties of the likelihood ratio test?

I am reading an article whose method is fully based on the likelihood ratio test. The author says that the LR test against one sided alternatives is UMP. He proceeds by claiming that "...even when ...
9
votes
6answers
2k views

Using p-value to compute the probability of hypothesis being true; what else is needed?

Question: One common misunderstanding of p-values is that they represent the probability of the null hypothesis being true. I know that's not correct and I know that p-values only represent the ...
7
votes
2answers
882 views

Is there a multiple testing problem when performing t-tests for multiple coeffcients in linear regression?

This question comes from a discussion on the recent post by @rvl It's all in the family; but do we include the in-laws too? Here's a common scenario that I've seen many times. A researcher runs a ...
7
votes
3answers
3k views

Is there a consensus on adjusting alpha for multiple contrasts if the main effect is significant?

After talking to a couple of statisticians, reading some book sections, internet articles, and forums like this, I am still unclear about multiplicity adjustments of post hoc tests and contrasts. Let'...
5
votes
2answers
305 views

Why do we trust the p-value when fitting a regression on a single sample?

I have code below that builds a linear model for a set of data: ...
5
votes
2answers
189 views

Appropriateness of one-sided hypothesis tests when testing medical treatments

I am working on some research where we are studying the effectiveness of two different types of brand new (i.e. never been studied before) treatments (call them high and low treatment levels) on death ...
4
votes
4answers
670 views

When a one-tailed test passes but a two-tailed test does not

(Sorry if this is obvious or is a duplicate. I couldn't find one.) Suppose two researchers are studying whether average height of some population has changed significantly. Researcher 1 hypothesizes ...
4
votes
2answers
9k views

How to set the significance level in a hypothesis test correctly

I have difficulties understanding probabilities. Let's imagine I have a test based on time series regressions and my null hypothesis is that I will pass my next exam in statistics. The test is ...
3
votes
3answers
10k views

Chi-Square-Test: Why is the chi-squared test a one-tailed test? [duplicate]

In my Masters thesis I conducted an experiment in that subjects should decide between two alternatives. I had a control and a treatment group. The hypothesis was that in the treatment group more ...
3
votes
2answers
2k views

How to define a rejection region?

This has always troubled me a bit. When I choose my hypothesis, do I define in some way the rejection region [RR], or, do I do that by choosing the test statistic I want to use? By fixing the ...

15 30 50 per page