Linked Questions

60
votes
13answers
16k views

If we fail to reject the null hypothesis in a large study, isn't it evidence for the null?

A basic limitation of null hypothesis significance testing is that it does not allow a researcher to gather evidence in favor of the null (Source) I see this claim repeated in multiple places, but I ...
0
votes
0answers
36 views

Interpreting hypothesis testing result (assuming that the null hypothesis is true) [duplicate]

I have a doubt on how to interpret a result of a hypothesis test. For example, a scenario where I have an existing configuration and also a new configuration. I am trying to check if with the new ...
0
votes
1answer
26 views

hypotheses testing [duplicate]

so I'm new to statistics and am not very comfortable with it yet. It may be a very simple question, but I'm finding this very difficult to understand In research papers the hypotheses are mostly in a ...
35
votes
7answers
6k views

Why is the null hypothesis often sought to be rejected?

I hope I am making sense with the title. Often, the null hypothesis is formed with the intention of rejecting it. Is there a reason for this, or is it just a convention?
17
votes
4answers
6k views

Misunderstanding a P-value?

So I've been reading a lot about how to correctly interpret a P-value, and from what I've read, the p-value says NOTHING about the probability that the null hypothesis is true or false. However, when ...
9
votes
6answers
2k views

Using p-value to compute the probability of hypothesis being true; what else is needed?

Question: One common misunderstanding of p-values is that they represent the probability of the null hypothesis being true. I know that's not correct and I know that p-values only represent the ...
3
votes
3answers
10k views

Chi-Square-Test: Why is the chi-squared test a one-tailed test? [duplicate]

In my Masters thesis I conducted an experiment in that subjects should decide between two alternatives. I had a control and a treatment group. The hypothesis was that in the treatment group more ...
11
votes
2answers
1k views

Family-wise error boundary: Does re-using data sets on different studies of independent questions lead to multiple testing problems?

If a team of researchers perform multiple (hypothesis) tests on a given data set, there is a volume of literature asserting that they should use some form of correction for multiple testing (...
7
votes
3answers
3k views

Is there a consensus on adjusting alpha for multiple contrasts if the main effect is significant?

After talking to a couple of statisticians, reading some book sections, internet articles, and forums like this, I am still unclear about multiplicity adjustments of post hoc tests and contrasts. Let'...
12
votes
1answer
5k views

Goodness-of-fit test in Logistic regression; which 'fit' do we want to test?

I am referring to the question and its answers: How to compare (probability) predictive ability of models developed from logistic regression? by @Clark Chong and answers/comments by @Frank Harrell. ...
4
votes
2answers
9k views

How to set the significance level in a hypothesis test correctly

I have difficulties understanding probabilities. Let's imagine I have a test based on time series regressions and my null hypothesis is that I will pass my next exam in statistics. The test is ...
1
vote
2answers
7k views

How to interpret MCAR (missing completely at random) any papers that I can read?

When dealing with missing data and Little's missing completely at random test, it's widely considered that if the test has a significance level of P>0.05 the data can be considered as MCAR. But, I ...
7
votes
2answers
886 views

Is there a multiple testing problem when performing t-tests for multiple coeffcients in linear regression?

This question comes from a discussion on the recent post by @rvl It's all in the family; but do we include the in-laws too? Here's a common scenario that I've seen many times. A researcher runs a ...
2
votes
1answer
5k views

Test if two normally distributed random variables have the same mean

We have two independent random variables which follow normal distributions $X_1\sim \mathcal N(\mu_1,\sigma_1)$, $X_2\sim \mathcal N(\mu_2,\sigma_2)$. From the context, we have that $\mu_1\leq\mu_2$. ...
4
votes
4answers
673 views

When a one-tailed test passes but a two-tailed test does not

(Sorry if this is obvious or is a duplicate. I couldn't find one.) Suppose two researchers are studying whether average height of some population has changed significantly. Researcher 1 hypothesizes ...
11
votes
1answer
959 views

What are the ''desirable'' statistical properties of the likelihood ratio test?

I am reading an article whose method is fully based on the likelihood ratio test. The author says that the LR test against one sided alternatives is UMP. He proceeds by claiming that "...even when ...
3
votes
2answers
2k views

How to define a rejection region?

This has always troubled me a bit. When I choose my hypothesis, do I define in some way the rejection region [RR], or, do I do that by choosing the test statistic I want to use? By fixing the ...
2
votes
3answers
230 views

Is this null hypothesis wrong?

The Wikipedia article Null hypothesis says in its opening paragraph: In inferential statistics, the term "null hypothesis" is a general statement or default position that there is no relationship ...
5
votes
2answers
309 views

Why do we trust the p-value when fitting a regression on a single sample?

I have code below that builds a linear model for a set of data: ...
2
votes
2answers
903 views

Choice of null and alternative hypothesis

A firm producing tobacco cigarettes claims that it has discovered a new technique for curing tobacco leaves, that results in an average nicotine content of a cigarette of less than 1.5 mg. To test ...
2
votes
2answers
257 views

Ambiguity of chi-squared test of association; how can the conclusion change using the ''same'' sample?

Recently I have been struggling with the following two questions regarding the chi-squared test of independence / association during reading an article. Is the chi-squared test able to detect no ...
2
votes
2answers
263 views

The rationale behind the “fail to reject the null” jargon in hypothesis testing?

In hypothesis testing with a Null and the Alternative hypothesis (assuming that these two cases are mutually inclusive of all the possible "truths"), we usually base our hypothesis selection criterion ...
5
votes
2answers
189 views

Appropriateness of one-sided hypothesis tests when testing medical treatments

I am working on some research where we are studying the effectiveness of two different types of brand new (i.e. never been studied before) treatments (call them high and low treatment levels) on death ...
2
votes
3answers
232 views

Null hypothesis for the one- and two-tailed test

From the Online Stat Book, Chapter 11.5: The null hypothesis for the two-tailed test is π = 0.5. By contrast, the null hypothesis for the one-tailed test is π ≤ 0.5. Why is that so? The null ...
1
vote
4answers
175 views

consequences of rejected/accepted hypothesis

A and B are some statements such that A implies B. I test the null hypothesis that A is true. If my test fails to reject A, does that result say anything about B? Analogously, if instead I test the ...
0
votes
2answers
143 views

What fraction to use in the formulation of a hypothesis test ? - Population fraction versus sample fraction

Imagine that I am studying racial discrimination. In particular, whether a white candidate (for a job) is chosen more often than an equally qualified minority candidate. I ask people to choose ...
0
votes
2answers
187 views

ANOVA for Regression

When using ANOVA to compare means, I understand the null hypothesis is u1=u2=u3.... as we need to combine all groups into a single group when assuming their means are the same. But for ANOVA for ...
0
votes
0answers
211 views

what “rejecting a null hypothesis” actually means?

I have a sample dateset consists two variables as "Score"(independent continuous) consists score of students in maths and school(dependent categorical)having 3 categories as "S1","S2","S3". I'd ...
3
votes
1answer
136 views

Probability of significant result in a second sample given first was significant

It's an example mentioned in Amos Tversky and Daniel Kahneman's Belief in the Law of Small Numbers: Suppose you have run an experiment on 20 subjects, and have obtained a significant result which ...
-1
votes
2answers
86 views

Definition of alternative hypothesis

Which of these properties is required of the alternative hypothesis, by definition, in the general case: a) If the alternative hypothesis is true, then the null hypothesis would necessarily tend ...
3
votes
0answers
89 views

How can we reject null hypothesis based on p-values? [closed]

In textbook statistical tests, we usually calculate the probability of observing the data we observed given that the null hypothesis is true, i.e. $P[D|H_0]$. If this probability is small (e.g. $<0....
0
votes
1answer
53 views

Power of a test in testing of hypothesis

Power in testing of hypothesis is defined as the probability of making the correct decision. Then why do text books describe power=1-P(type 2 error) and not 1-P(type 1 error)
1
vote
0answers
50 views

What basis should be used to devize a significance/hypothesis test?

I am trying to understand the basis of hypothesis testing and I came up with a paradox. Let me explain with an example. Consider the case of trying to determine whether is coin is fair. The null ...