# Questions tagged [type-i-and-ii-errors]

Type I: rejecting the null hypothesis when it is true. Type II: not rejecting the null hypothesis when the alternative is true.

301 questions
Filter by
Sorted by
Tagged with
13 views

### Calculate the propagated uncertainty by measurement error in linear fit coefficients

I am currently taking a physic experiment class and am required to analyse the uncertainty of data caused by measurement error. They have provided me the standard way to propagate the error, namely by ...
1 vote
46 views

### Test whether my new method's type I error rate correctly matches chosen alpha

I am testing two different methods for hypothesis testing, bootstrapped likelihood ratio tests, and likelihood ratio tests assuming an asymptotic distribution. I would like to see if the number of ...
147 views

### Why do we compute the standard deviation of the proportion using the value assumed under the null hypothesis?

In type 2 error computation, why do we compute the standard deviation using the value assumed under the null hypothesis even if we provide the alternative hypothesis as possible true value? If in null ...
• 33
55 views

### type I error in multiple comparisons summary statistics

I understand type I error can increase if we run 'multiple comparisons'. But does that refer to comparisons within a multi-category variable (or a group variable)? or multiple analyses using the same ...
• 191
66 views

### Probability of committing a type I error

Let $X_1, ... ,X_n$ be an iid sample from the Bern(p) distribution, with $p \in (0, 1)$. I want to test the null hypothesis $H_0: p = \frac{1}{2}$ against the alternative $H_1: p \in (0, 1) \setminus$...
• 117
116 views

### If we know that there were no type 1 errors in previous tests, should we still make corrections for multiplicity?

Suppose there's a study where an interim analysis was performed to evaluate efficacy with the intention of stopping the study. However, since the predefined success criterion wasn't achieved, the ...
• 2,207
26 views

### In a mixed effects model, can leaving out the intercept-slope correlation parameter inflate type I error?

I am considering leaving out the intercept-slope correlation parameter in a mixed effects model to avoid convergence issues (i.e., in nlme::lme, ...
• 225
1 vote
40 views

### Robust standard errors leading to false positives [closed]

I have an odd scenario in my data analysis and I'm not sure what is causing it. I have a large set of tuples $(Y_1, X_i) \dots, (Y_N, X_N)$ where $Y_i$ is a random vector from some arbitrary ...
• 147
28 views

### Should I normalise my dependent variable by a measured variable that I want to be constant?

This is a question about the validity of applying the results of a statistical test, and unfortunately will require some explanation of the experiment in question. Essentially, I want to measure how ...
1 vote
87 views

### Mann-Whitney U test, compared to Binomial test

I was doing some simulations on the Mann-Whitney U test. I collected a number of U statistics (U is the smallest number of wins out of all pairwise contests between 2 samples: compare all values of ...
• 1,994
1 vote
69 views

### Should we favor family-wise error rate control or statistical power?

To counteract the multiple comparison problem, people generally use procedures that ensure strong level alpha FWER control (e.g., Bonferroni, Holm). However, these procedures necessarilly produce a ...
• 191
57 views

### Distribution for which one-sample t-test yields inflated type 1 error rates

Type 1 error rates of the one-sample t-test (two-tailed) can be inflated for example for data from the exponential distribution. However, I am looking for distributions which yield inflated type 1 ...
1 vote
64 views

### Can we derive type 2 error probability from confidence interval?

If we can compute confidence intervals for an estimate $\hat{\theta}$, then we can also compute the p value for an associated hypothesis $\theta=\theta_0$, which is the probability of a type 1 error (...
• 1,152
90 views

### Bayesian stats and multiple tests

Are Bayesian models subject to the same problems as frequentist ones, where we cannot run a bunch of different models due to Type I error? For example, let's say I have a large data frame on airplanes,...
• 89
1 vote
63 views

### How to perform a 2 sample Kolmogorov-Smirnov test controlling for a type II error instead of type 1

I have four completely different sets of two samples and want to check for each set if both samples come from the same distribution, so four Kolmogorov-Smirnov tests total. If it's going to be ...
• 21
27 views

### Is there any valid approach to adjust the statistical power taking the inflation of type 1 error into account?

I'm conducting some simulation analysis under different correlation structures where H0: absence of interaction component in the model. I computed type 1 error rate under the null hypothesis at some ...
122 views

### The use of p-values in tests of d-separation

In his very helpful online book on structural equation modeling, Jon Lefcheck writes the following concerning d-separation tests for SEMs: Once the model is fit, statistical independence is assessed ...
• 371
201 views

### On the logic meaning behind rejecting the null hypothesis and its relantionship with Type I error

I understand that the p-value is defined as the probability to obtain a "more extreme" value of $w$ if $H_0$ is true, i.e. $p=P(|W| > |w| \ |H_0$is true$)$ and the "significance ...
• 747
1 vote
42 views

• 2,966
38 views

### Methods to control for false omission rate?

What methods are available to control for false omission rate in multiple comparisons, similar to the way Benjamini-Hochberg controls for false discovery rate? The goal is to filter out obvious ...
• 253
1 vote
124 views

• 1,676
623 views

### For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?

This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size? In the referenced post, we see that the bootstrap approach does not control type-1 ...
• 21
368 views

### Is confounding a source of Type I errors?

I would be interested to explore the potential sources of Type I errors in randomised controlled trials. A key question in this respect is whether confounding can be understood as one such source.
• 31
391 views

### Is my way of deriving a statistical test from Hoeffding's inequality correct?

I'm trying to deduce from samples of observations from two finite sets of random variables $X_{1}, ..., X_{n}$ and $Y_{1}, ..., Y_{m}$ that the expected values of the average of those random variables ...
• 155
53 views

### Testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$: Find $c$ in rejection region so that probability of a type I error is $\alpha$

My problem is as follows: Let $X_1,\ldots,X_n$ be a random sample from a $N(\theta,1)$ distribution, and consider testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$. Let \$\Omega = \{\mathbf{X}: ...
• 121
215 views

### How to simulate type I error for random-effects model?

Building upon this post How to simulate type I error and type II error. I would like to simulate type I error for a random-effects model I generated. The statistic of interest is standard deviations ...
• 211
360 views

### simple vs simple hypothesis, finding sample size given type 1 and 2 error probabilities

The problem is: It is decided to investigate the hypotheses H0 : μ = €678 and H1 : μ = €720. Assuming that the prices are normally distributed with standard deviation €100, what sample size must be ...