Hot answers tagged hypothesis-testing
7
Yes Neyman Pearson Lemma can apply to the case when simple null and simple alternative don't belong to the same family of distributions.
Let we want to construct a Most Powerful(MP) test of $H_0:X\sim N(0,1)$ against $H_1 : X\sim \text{Exp}(1)$ of its size.
For a particular $k$, our critical function by Neyman Pearson lemma is
$$\phi(x) =\begin{cases} ...
6
Let's find out whether this is a good test or not. There's a lot more to it than just claiming it's bad or showing in one instance that it doesn't work well. Most tests work poorly in some circumstances, so often we are faced with identifying the circumstances in which any proposed test might possibly be a good choice.
Description of the test
Like any ...
6
With increasing sample size, the statistical power (see below) to detect even the smallest effect size is also increasing and these tiny effect sizes are then found to be statistically significant, even though they bear no relevance at all. Just as a thought experiment to illustrate it further: What if you could include all people of interest in a study. All ...
5
Answers to question 1,2,3,4 ($Z$-test)
The decreasing link between the $p$-value and the observed power is intuitively highly expected: the $p$-value $p^{\text{obs}}$ is low when the observed sample mean $\bar y^{\text{obs}}$ is high ($H_1$ favoured), and since $\bar y^{\text{obs}} = \hat\mu$ the observed power is high because the power function $\mu ...
4
Often there are several statistics that will all result in the same p-value/result. For example in a 2 sample case the difference of the 2 means, the mean of group A, and the sum of the values in group A will all result in the same p-value (this is because given the data values and sample sizes you can calculate the 1st 2 given only the 3rd). I would ...
4
You've got a couple things confused. First, we never *accept $H_0$", we either reject it or fail to reject it.
Second, if you want to word it in terms of P, then the procedure is always:
If $P_{\text{observed}} < P_{\text{critical}}$ then reject $H_0$, otherwise, fail to reject.
but if you word it in terms of test statistics then it's the other way ...
4
The likelihood ratio test is considered the gold standard as it measures the difference in likelihood between the null value and the ML estimate. It also only requires you to calculate the value of the likelihood (or log-likelihood), though you will often calculate or estimate the 1st and/or second derivatives to find the ML estimate.
The Wald statistic ...
4
Q2. The likelihood ratio's a sensible enough test statistic but (a) the Neyman-Pearson Lemma doesn't apply to composite hypotheses, so the LRT won't necessarily be most powerful; & (b) Wilks' Theorem only applies to nested hypotheses, so unless one family is a special case of the other (e.g. exponential/Weibull, Poisson/negative binomial) you don't know ...
3
No, correlation is not a good test of this.
x <- 1:100 #Uniform
y <- sort(rnorm(100)) #Normal
cor(x,y) #.98
I don't know of a good test that compares whether, e.g. two distributions are both normal, but possibly with different mean and s.d. Indirectly, you could test the normality of each, separately, and if both seemed normal, guess they both were. ...
3
For concreteness, imagine a one sample test of means (large sample, on something where the population mean and variance exists to make the argument a little simpler).
Let the difference between the true mean and the hypothesized sample mean be any nonzero $\delta$. Then the sampling distribution of the sample mean minus the hypothesized mean will itself ...
3
I don't think there can be a general and precise answer. There can be general answers that are loose, and specific answers that are precise.
Most generally (and most loosely) an effect size is a statistical measure of how big some relationship or difference is.
In regression type problems, one type of effect size is a measure of how much of the dependent ...
2
The information provided does not indicate fraud by either party. Assuming 95% confidence as the acceptance point then 2 in 20 tests could be expected to produce a type 1 or 2 error. This is a major reason for doing meta analyses.
Furthermore, to check for fraud, one would want to review the record keeping for the two experiments and perhaps corroborative ...
2
Similar to Stijn's comment, there isn't sufficient information to make a conclusive judgment. There could be fraud (in either of the papers, not just the first), or there could have been a Type I / Type II error in one of the papers.
Out of interest if the two studies were identical and independent, so had identical power functions which factored, would it ...
2
There appears to be a difference in the interpretation of a statistical formula. One quick, simple, and compelling way to resolve such differences is to simulate the situation. Here, you have noted there will be a difference when the players play different numbers of games. Let's therefore retain every aspect of the question but change the number of games ...
2
Spearman rank correlation is just Pearson correlation applied to ranks, a point often obscured by the emphasis on the simple computational short-cut formula for Spearman that is found in many books. So, I wouldn't rule out Fisher's z procedures for Spearman. There is a caution that the sampling distribution will differ at least a bit with Spearman -- as the ...
2
I was wondering what it means by a variance being explained or unexplained?
It the context of ANOVA it means the variance "explained" by group membership and the variance that remains unexplained. To understand this in detail you have to really look at the equations. I'll try to explain it anyway without introducing too many equations. In the case of a ...
1
The purpose of of multiple comparisons procedures is not to test the overall significance, but to test individual effects for significance while controlling the experimentwise error rate. It's quite possible for e.g. an omnibus F-test to be significant at a given level while none of the pairwise Tukey tests are—it's discussed here & here.
...
1
I wouldn't look for a rigorous definition of omnibus test. It seems typically used for overall tests with wide scope, packing several tests into one.
Other terms used with similar import are portmanteau statistic and factotum statistic.
Over a century or more, there have been all sorts of fashions over terminology, including statisticians reaching for ...
1
And do you have reason to believe that this data would show positive or negative autocorrelation? i.e. do you need to do a one or two sided test? Often a two sided DW test is simply carried out as two one sided tests and the $\alpha$ (type I error) for the two sided is simply doubled. So for example for the test:
$H_o: \rho = 0$
$H_a: \rho \neq 0$
...
1
You choose a test statistic that measures what you're interested in/has the properties you need.
If you want to compare means, you base it of differences of means; if you want a robust comparison of location, you measure something else; if you want to compare standard deviations, you use a statistic that does that; if you want to compare all aspects of the ...
1
Here is a partial answer, maybe someone could complement it.
"Statistical sufficency" means that no other statistic uses more information from the sample. Definition of sufficiency. Maximum likelihood estimate $\hat{\theta}$ is a sufficient statistic.
Cramer-Rao lower bound states the lowest value (lower bound) for the variance. It is computed from the ...
1
Not sure if this is an answer. But perhaps a few comments. If I am restating what you are probably already aware of, my apologies.
First, based on the Fisher–Neyman Factorization, if $T(\mathbf{x})$ is a sufficient statistic, then the likelihood function factorizes to the product of (1) a function that does not involve $\theta$; times (2) a function that ...
1
Seems to me the answer is more in the game theory of research than in the statistics. Having 1% and 5% burned into the general consciousness means that researchers aren't effectively free to choose significance levels that suit their predispositions. Say we saw a paper with a p-value of .055 and where the significance level had been set at 6% - questions ...
Only top voted, non community-wiki answers of a minimum length are eligible
