# Questions tagged [type-i-and-ii-errors]

Type I: rejecting the null hypothesis when it is true. Type II: not rejecting the null hypothesis when the alternative is true.

298 questions
Filter by
Sorted by
Tagged with
46 views

### type I error in multiple comparisons summary statistics

I understand type I error can increase if we run 'multiple comparisons'. But does that refer to comparisons within a multi-category variable (or a group variable)? or multiple analyses using the same ...
• 191
65 views

### Probability of committing a type I error

Let $X_1, ... ,X_n$ be an iid sample from the Bern(p) distribution, with $p \in (0, 1)$. I want to test the null hypothesis $H_0: p = \frac{1}{2}$ against the alternative $H_1: p \in (0, 1) \setminus$...
• 117
113 views

### If we know that there were no type 1 errors in previous tests, should we still make corrections for multiplicity?

Suppose there's a study where an interim analysis was performed to evaluate efficacy with the intention of stopping the study. However, since the predefined success criterion wasn't achieved, the ...
• 2,245
26 views

### In a mixed effects model, can leaving out the intercept-slope correlation parameter inflate type I error?

I am considering leaving out the intercept-slope correlation parameter in a mixed effects model to avoid convergence issues (i.e., in nlme::lme, ...
• 225
1 vote
34 views

### Robust standard errors leading to false positives [closed]

I have an odd scenario in my data analysis and I'm not sure what is causing it. I have a large set of tuples $(Y_1, X_i) \dots, (Y_N, X_N)$ where $Y_i$ is a random vector from some arbitrary ...
• 157
28 views

### Should I normalise my dependent variable by a measured variable that I want to be constant?

This is a question about the validity of applying the results of a statistical test, and unfortunately will require some explanation of the experiment in question. Essentially, I want to measure how ...
1 vote
71 views

### Mann-Whitney U test, compared to Binomial test

I was doing some simulations on the Mann-Whitney U test. I collected a number of U statistics (U is the smallest number of wins out of all pairwise contests between 2 samples: compare all values of ...
• 837
1 vote
54 views

### Should we favor family-wise error rate control or statistical power?

To counteract the multiple comparison problem, people generally use procedures that ensure strong level alpha FWER control (e.g., Bonferroni, Holm). However, these procedures necessarilly produce a ...
• 191
52 views

### Distribution for which one-sample t-test yields inflated type 1 error rates

Type 1 error rates of the one-sample t-test (two-tailed) can be inflated for example for data from the exponential distribution. However, I am looking for distributions which yield inflated type 1 ...
1 vote
60 views

### Can we derive type 2 error probability from confidence interval?

If we can compute confidence intervals for an estimate $\hat{\theta}$, then we can also compute the p value for an associated hypothesis $\theta=\theta_0$, which is the probability of a type 1 error (...
• 1,148
88 views

### Bayesian stats and multiple tests

Are Bayesian models subject to the same problems as frequentist ones, where we cannot run a bunch of different models due to Type I error? For example, let's say I have a large data frame on airplanes,...
• 89
52 views

### How to perform a 2 sample Kolmogorov-Smirnov test controlling for a type II error instead of type 1

I have four completely different sets of two samples and want to check for each set if both samples come from the same distribution, so four Kolmogorov-Smirnov tests total. If it's going to be ...
• 11
27 views

### Is there any valid approach to adjust the statistical power taking the inflation of type 1 error into account?

I'm conducting some simulation analysis under different correlation structures where H0: absence of interaction component in the model. I computed type 1 error rate under the null hypothesis at some ...
112 views

### The use of p-values in tests of d-separation

In his very helpful online book on structural equation modeling, Jon Lefcheck writes the following concerning d-separation tests for SEMs: Once the model is fit, statistical independence is assessed ...
• 371
196 views

### On the logic meaning behind rejecting the null hypothesis and its relantionship with Type I error

I understand that the p-value is defined as the probability to obtain a "more extreme" value of $w$ if $H_0$ is true, i.e. $p=P(|W| > |w| \ |H_0$is true$)$ and the "significance ...
• 747
1 vote
38 views

• 2,884
37 views

### Methods to control for false omission rate?

What methods are available to control for false omission rate in multiple comparisons, similar to the way Benjamini-Hochberg controls for false discovery rate? The goal is to filter out obvious ...
• 253
1 vote
123 views

• 1,686
597 views

### For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?

This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size? In the referenced post, we see that the bootstrap approach does not control type-1 ...
• 21
357 views

### Is confounding a source of Type I errors?

I would be interested to explore the potential sources of Type I errors in randomised controlled trials. A key question in this respect is whether confounding can be understood as one such source.
• 31
372 views

### Is my way of deriving a statistical test from Hoeffding's inequality correct?

I'm trying to deduce from samples of observations from two finite sets of random variables $X_{1}, ..., X_{n}$ and $Y_{1}, ..., Y_{m}$ that the expected values of the average of those random variables ...
• 155