Hot answers tagged

44

Traditionally, the null hypothesis is a point value. (It is typically $0$, but can in fact be any point value.) The alternative hypothesis is that the true value is any value other than the null value. Because a continuous variable (such as a mean difference) can take on a value which is indefinitely close to the null value but still not quite equal and ...


28

Consider the case where the null hypothesis is that a coin is 2 headed, i.e. the probability of heads is 1. Now the data is the result of flipping a coin a single time and seeing heads. This results in a p-value of 1.0 which is greater than every reasonable alpha. Does this mean that the coin is 2 headed? it could be, but it could also be a fair coin and ...


9

Ok, here's my first attempt. Close scrutiny and comments appreciated! The Two-Sample Hypotheses If we can frame two-sample one-sided Kolmogorov-Smirnov hypothesis tests, with null and alternate hypotheses along these lines: H$_{0}\text{: }F_{Y}\left(t\right) \geq F_{X}\left(t\right)$, and H$_{\text{A}}\text{: }F_{Y}\left(t\right) < F_{X}\left(t\right)$,...


8

The logic of TOST employed for Wald-type t and z test statistics (i.e. $\theta / s_{\theta}$ and $\theta / \sigma_{\theta}$, respectively) can be applied to the z approximations for nonparametric tests like the sign, sign rank, and rank sum tests. For simplicity I assume that equivalence is expressed symmetrically with a single term, but extending my answer ...


7

I have recently thought about an alternative way of "equivalence testing" based on a distance between the two distributions rather than between their means. There are some methods providing confidence intervals for the overlap of two Gaussian distributions: The overlap $O(P_1,P_2)$ of (between?) two distributions $P_1$ and $P_2$ has a nice probabilistic ...


7

The short answer is yes, you can do it, since the TOST methodology is not restricted to t-tests. The p-value is the larger of the two p-values. A quick Google search led me to a methodological article (Meier U. Nonparametric equivalence testing with respect to the median difference. Pharm Stat. 2010 Apr-Jun;9(2):142-50) describing this procedure in detail.


7

Absence of evidence is not evidence of an absence (the title of an Altman, Bland paper on BMJ). P-values only give us evidence of an absence when we consider them significant. Otherwise, they tell us nothing. Hence, absence of evidence. In other words: we don't know and more data may help.


7

While one can use the t test to test for proportion difference, the z test is a tad more precise, since it uses an estimate of the standard deviation formulated specifically for binomial (i.e. dichotomous, nominal, etc.) data. The same applies to the z test for proportion equivalence. First, the z test for difference in proportions of two independent ...


6

Your logic applies in exactly the same way to the good old one-sided tests (i.e. with $x=0$) that may be more familiar to the readers. For concreteness, imagine we are testing the null $H_0:\mu\le0$ against the alternative that $\mu$ is positive. Then if true $\mu$ is negative, increasing sample size will not yield a significant result, i.e., to use your ...


6

We never "accept the null hypothesis" (without also giving consideration to power and minimum relevant effect size). With a single hypothesis test, we pose a state of nature, $H_{0}$, and then answer some variation of the question "how unlikely are we to have observed the data underlying our test statistic, assuming $H_{0}$ (and our distributional assumption)...


6

An alternative to TOST in equivalence testing is based on the confidence interval approach: Let $\Delta$ denote the prespecified equivalence margin and $$ \theta := \sup_t |F_X(t) - F_Y(t)| $$ the Kolmogorov-Smirnov distance between the unknown underlying distribution functions. Now, if a 90% confidence interval for $\theta$ is completely within $[-\...


6

Regression table presentations are easy enough to modify to accommodate tests for equivalence, including relevance tests—where you base conclusions off of both tests for difference (tests of $H^{^{+}}_{0}$) and tests for equivalence (tests of $H^{^{-}}_{0}$). For example (assuming you are presenting multiple tests in a regression context, hence the $\beta$): ...


5

The $1-2\alpha$ is not because you calculate the CI for each group separately. It is because you calculate the "inequivalence" to the upper and to the lower end separately. The parameter $\theta$ lies in the equivalence interval $[\epsilon_L, \epsilon_U]$ iff $$\theta \geq \epsilon_L \wedge \theta \leq \epsilon_U.$$ Each part is tested separately by a one ...


5

The null hypothesis, $H_0$, is usually taken to be the thing you have reason to assume. Often times it is the "current state of knowledge" that you wish to show is statistically unlikely. The usual set-up for hypothesis testing is minimize type I error, that is, minimize the chance that we reject the null hypothesis in favor of the alternative $H_1$ even ...


4

Yes. This is equivalence testing. Basically you reverse the null and alternative hypothesis and base the sample size on the power to show that the difference of the means is within the window of equivalence. Blackwelder called it "Proving the null hypothesis." This is commonly done in pharmaceutical clinical trials where equivalence of a generic drug to ...


3

I think one can do all these multiple tests of equivalence within a single linear-mixed models. Given you have multiple (2+) measures after the change took place it is rather natural to present these multiple tests as part of a single repeated-measurements model. In particular, one could define indicator variables between the successive steps and then check ...


3

Very interesting question!! You are using the logical consequence, i.e., the entailment condition. This entailment condition forms the very basis of the classical logic, it guarantees the inference or deduction of a result from a premise. The reasoning behind your proposal follows: If $H_0$ entails $H_0'$, then the observed data should draw more evidence ...


3

It's not a TOST per se, but the Komolgorov-Smirnov test allows one to test for the significance of the difference between a sample distribution and a second reference distribution you can specify. You can use this test to rule out a specific kind of different distribution, but not different distributions in general (at least, not without controlling for ...


3

First question: UMP is, nomen es omen, most powerful. If both the sample size and the equivalence region are small, it may happen to the TOST that confidence intervals will hardly ever fit into the equivalence region. This results in nearly zero power. Also, the TOST is generally conservative (even with an $1-2\alpha$ confidence interval). Whenever the UMP ...


3

When conducting the Kolmogorov-Smirnov test, we assume $H_0:$ the two distributions are equivalent. We then calculate a test statistic and, if the corresponding $p$-value is small enough, we reject $H_0$ and conclude $H_A:$ the two distributions are different. As far as hypothesis tests go, we use a $p$-value to quantify the amount of evidence we have to ...


2

These arguments from "Bayesian Estimation Supersedes the t-Test" seem relevant: This article introduces an intuitive Bayesian approach to the analysis of data from two groups. In particular, the analysis reveals the relative credibility of every possible difference of means, every possible difference of standard deviations, and all possible effect sizes. ...


2

I don't know any specific references for this case. In analogy to some of the methods for repeated measures ANOVA, the relevant t-test would use the mean of the two 'after' observations and compare it with the 'before' observation. The variance of the average within difference will be smaller with more observations per individual, so the test still takes ...


2

Since no one answered, I will try to show what I understood and how it should be choosen the region of similarity. The null hypothesis is $H_0 : |\mu_1 - \mu_2| > \varepsilon$ (where $\mu_1$ and $\mu_2$ are means for controller 1 and 2). Thus $\varepsilon$ is the minimum value by which we will define how similar is the response (tracking error) of the ...


2

As I understand it, your model is: $$\text{SEND} = \beta_{0} + \beta_{pt}pt + \beta_{partner}partner + \beta_{town}town + \mathbf{B}_{controls}\mathbf{controls} + \varepsilon$$ So your estimated effect of treatment on SEND is given by $\hat{\beta}_{pt}$. Tests for difference are reported in the vanilla output for linear regression in Stata: To the right ...


1

I was also wondering if an epsilon of 2 sets a margin of 2 above and 2 below or a margin of 1 above and 1 below for a total rang of 2. As I couldn't find the answer nor understand the R code I contacted directly the author of the package "equivalence", professor Robinson, who very kindly answered: An epsilon of 2 sets margin for 2 above and 2 below.


1

I think it is a synonym to the equivalence margin. Because there is no exact equivalence it is a range of the similarity. Here is a article which describes the Equivalence and Noninferiority Testing. EDIT: For the hypothesis testing it is necessary to set a acceptable range of unequality. The deviation of the equal margin can be in positive and negative ...


1

Normally with repeated outcome variables one would use a mixed effects ANOVA. The baseline assessment is considered fixed, or given, so it's simply a matter of using an offset. The inference is then based around the intercept, $\beta_0$ of the following linear model: \begin{equation} X^{b,i} - \mbox{offset}(X^{a, i}) = \beta_0 + \gamma_i \end{equation} ...


Only top voted, non community-wiki answers of a minimum length are eligible