Hot answers tagged hypothesis-testing
9
If all the assumptions hold and you have the correct form for $R^2$ then the usual F statistic can be computed as $F = \frac{ R^2 }{ 1- R^2} \times \frac{ \text{df}_2 }{ \text{df}_1 }$. This value can then be compared to the appropriate F distribution to do an F test. This can be derived/confirmed with basic algebra.
8
One idea would be to calculate the log-loss of the probabilities, relative to the outcome. If the log-loss is low, the probabilities closely match the outcome. Another idea would be to bin the probabilities into quartiles or deciles, and find the average probability and average outcome in each bin.
Can you share the dataset? It seems very interesting.
...
7
Yes Neyman Pearson Lemma can apply to the case when simple null and simple alternative don't belong to the same family of distributions.
Let we want to construct a Most Powerful(MP) test of $H_0:X\sim N(0,1)$ against $H_1 : X\sim \text{Exp}(1)$ of its size.
For a particular $k$, our critical function by Neyman Pearson lemma is
$$\phi(x) =\begin{cases} ...
6
The KPSS test is used for testing a null hypothesis that an observable time series is stationary around a deterministic trend. You can see that the critical values are given by:
Critical values for H0: mIlliq1 is trend stationary
So as you can see, you cannot reject the H0, that your data is trend-stationary. So the data follow a straight-line time trend ...
6
With increasing sample size, the statistical power (see below) to detect even the smallest effect size is also increasing and these tiny effect sizes are then found to be statistically significant, even though they bear no relevance at all. Just as a thought experiment to illustrate it further: What if you could include all people of interest in a study. All ...
5
Recall that in a regression setting, the F statistic is expressed in the following way.
$$
F = \frac{(TSS - RSS)/(p-1)}{RSS/(n-p)}
$$
where TSS = total sum of squares and RSS = residual sum of squares. This is an F distribution with degrees of freedom p-1 and n-p.
Also recall that
$$
R^2 = 1 - \frac{RSS}{TSS} = \frac{TSS - RSS}{TSS}
$$
simple algebra ...
5
The reason you don't have the same epiphanic look on your face as that guy is I think that . . . the statement isn't true.
A null hypothesis is the hypothesis that any difference between the control and experimental conditions is due to chance.
An uninformative prior is meant to state that you have prior data on a question, but that it doesn't tell you ...
4
(To make our notions a little more precise, let's call the 'test statistic' the distribution of the thing we look up to actually compute the p-value. This means that for a two-tailed t-test, our test statistic would be $|T|$ rather than $T$.)
What a test statistic does is induce an ordering on the sample space (or more strictly, a partial ordering), so that ...
4
You did not specify your variables, if they are binary or something else. I think you talk about binary variables. There also exist multinomial versions of the probit and logit model.
In general, you can use the complete trinity of test approaches, i.e.
Likelihood-Ratio-test
LM-Test
Wald-Test
Each test uses different test-statistics. The standard ...
4
Answers to question 1,2,3,4 ($Z$-test)
The decreasing link between the $p$-value and the observed power is intuitively highly expected: the $p$-value $p^{\text{obs}}$ is low when the observed sample mean $\bar y^{\text{obs}}$ is high ($H_1$ favoured), and since $\bar y^{\text{obs}} = \hat\mu$ the observed power is high because the power function $\mu ...
4
Q2. The likelihood ratio's a sensible enough test statistic but (a) the Neyman-Pearson Lemma doesn't apply to composite hypotheses, so the LRT won't necessarily be most powerful; & (b) Wilks' Theorem only applies to nested hypotheses, so unless one family is a special case of the other (e.g. exponential/Weibull, Poisson/negative binomial) you don't know ...
4
The likelihood ratio test is considered the gold standard as it measures the difference in likelihood between the null value and the ML estimate. It also only requires you to calculate the value of the likelihood (or log-likelihood), though you will often calculate or estimate the 1st and/or second derivatives to find the ML estimate.
The Wald statistic ...
4
You've got a couple things confused. First, we never *accept $H_0$", we either reject it or fail to reject it.
Second, if you want to word it in terms of P, then the procedure is always:
If $P_{\text{observed}} < P_{\text{critical}}$ then reject $H_0$, otherwise, fail to reject.
but if you word it in terms of test statistics then it's the other way ...
3
I'm the one that created the graphic, though as noted in the accompanying post it's not originally my insight. Let me provide some context for how it came up and do my best to explain how I understand it. The realization occurred during a discussion with a student who had mostly learned the Bayesian approach to inference up to that point. He was having a ...
3
The standard approaches are the Wald test, the likelihood ratio test and the score test. Asymptotically they should be the same. In my experience the likelihood ratio tests tends to perform slightly better in simulations on finite samples, but the cases where this matters would be in very extreme (small sample) scenarios where I would take all of these tests ...
3
Wald test
One standard approach is the Wald test. This is what the Stata command test does after a logit or probit regression. Let's see how this works in R by looking at an example:
mydata <- read.csv("http://www.ats.ucla.edu/stat/data/binary.csv") # Load dataset from the web
mydata$rank <- factor(mydata$rank)
mylogit <- glm(admit ~ gre + gpa + ...
3
It is not quite true that the rejection regions can be "chosen" to be nested or not. For simple hypotheses, and a continuous test statistic, the rejections regions of maximal power tests are surely nested via the Neyman-Pearson Lemma. The same goes for composite hypotheses and UMP tests.
GLR tests do not guarantee maximal power, but I do not know any ...
3
Technically, any ordinal scaling will do, but you can't take the means (or do t-test). 1, 2, 3? Fine. Also, per Stevens' ideas, 1, 2, 12o9101, or 1, 1.2, 2.1021. This points to one of the problems with Stevens' classification. Many things are in-between ordinal and interval.
You could do a Jonckeere Terpstra test; this tests for differences where one ...
3
For concreteness, imagine a one sample test of means (large sample, on something where the population mean and variance exists to make the argument a little simpler).
Let the difference between the true mean and the hypothesized sample mean be any nonzero $\delta$. Then the sampling distribution of the sample mean minus the hypothesized mean will itself ...
2
$|X_{1,1} - \mu|$ cannot have a hypergeometric distribution in general because $\mu$ does not need to be an integer value and then $|X_{1,1} - \mu|$ would not be an integer. But conditionally on the margins, $X_{1,1}$ will have a hypergeometric distribution.
If you do it properly and fix the margins to known values, you can consider $X_{1,1}$ (or any other ...
2
Point hypothesis are not common(reasonable) in a Bayesian framework since their probability, under a continuous model, is zero.
Something that seems to fit in your context is the calculation of the posterior stress-strength coefficient
$$\theta=P(\lambda_1\lt \lambda_2) = \int_{0}^{\infty}\int_{0}^{l_2}\pi_{\lambda_1\vert{\bf ...
2
The null hypothesis isn't equivalent to a Bayesian uninformative prior for the simple reason that Bayesians can also use null hypotheses and perform hypothesis tests using Bayes' factors. If they were equivalent, Bayesians wouldn't use null hypotheses.
However, both frequentist and Bayesian hypothesis testing incorporate an element of self-skepticism, in ...
2
See this Wikipedia article:
For the case of a single parameter and data that can be summarized in
a single sufficient statistic, it can be shown that the credible
interval and the confidence interval will coincide if the unknown
parameter is a location parameter (...) with a prior that is a uniform
flat distribution (...) and also if the unknown ...
2
From a frequentist perspective you should calculate the standard error of the sample mean (using the known population standard deviation). You can then work out how many standard errors the sample mean is away from six ppb and look at the corresponding Normal probability density function tails (note the plural).
The assumption is that there's a fixed lead ...
2
The asymptotic Wald test is what you want. You want to calculate:
$W=\left(\hat{\beta}_1-\hat{\beta}_2 \right)'\left( V(\hat{\beta}_1-\hat{\beta}_2 )\right)^{-1} \left( \hat{\beta}_1-\hat{\beta}_2 \right)$
How to calculate the variance in the middle, though? Write the difference in the beta hats like this:
$\begin{align}
\hat{\beta}_1-\hat{\beta}_2 ...
2
Your interpretation of a low p-value is a very common one, but it is not quite correct. The phrase "is unlikely to be equal" has no meaning in a Frequentist context. Either all of the population means are equal, or they are not. To the extent that something like a statement of probability can be made in this situation, the correct statement would be ...
2
I can't speak to what the people setting the exam might do; sometimes the actuarial choices on statistical matters baffle me (I have an actuarial background as well as a stats one, so I have some exposure to these).
I can only speak to what I see as the statistics issues.
Given 20 is pretty much near the middle of the null distribution, which itself is ...
2
You have a couple of options. One is to do a test (or confidence interval) for the difference of two proportions. Here is a link to a similar question with a solution in R: Change in binomial proportion confidence interval
Another option is to do a permutation test since your samples are so small.
2
Since your dependent variable is binary (yes/no), a good starting place at least is logistic regression with coupon-type as a categorical independent variable; in the second case, you would have two independent variables. However, the second case might involve dependent data, which would make things more complex. This would be the case if, as seems likely, ...
2
There appears to be a difference in the interpretation of a statistical formula. One quick, simple, and compelling way to resolve such differences is to simulate the situation. Here, you have noted there will be a difference when the players play different numbers of games. Let's therefore retain every aspect of the question but change the number of games ...
Only top voted, non community-wiki answers of a minimum length are eligible
