Questions tagged [type-i-and-ii-errors]
Type I: rejecting the null hypothesis when it is true. Type II: not rejecting the null hypothesis when the alternative is true.
292
questions
1
vote
0
answers
22
views
Is there any valid approach to adjust the statistical power taking the inflation of type 1 error into account?
I'm conducting some simulation analysis under different correlation structures where H0: absence of interaction component in the model.
I computed type 1 error rate under the null hypothesis at some ...
2
votes
1
answer
88
views
The use of p-values in tests of d-separation
In his very helpful online book on structural equation modeling, Jon Lefcheck writes the following concerning d-separation tests for SEMs:
Once the model is fit, statistical independence is assessed ...
- 329
4
votes
1
answer
190
views
On the logic meaning behind rejecting the null hypothesis and its relantionship with Type I error
I understand that the p-value is defined as the probability to obtain a "more extreme" value of $w$ if $H_0$ is true, i.e.
$p=P(|W| > |w| \ |H_0 $is true$)$
and the "significance ...
- 747
1
vote
0
answers
36
views
Magnitude of Type I error inflation related to error noise after model selection
I am investigating Type I error inflation for the one-sided test $$H_0:\beta_2=0$$ after one step of forward stepwise regression for the following model: $$Y=X_1+\beta_2X_2+\epsilon$$ where $\epsilon \...
- 11
0
votes
1
answer
45
views
How can I compute the probability of committing a type 2 error?
Let $X_1,...X_n$ be an iid sample distributed as $\mathcal{N}(\mu,1)$. We have the following tests $$H_0: \mu=0~~~\text{vs.}~~~H_1:\mu\neq 0$$and we use the statistic $T=\frac{1}{n}\sum_{k=1}^n X_k$ ...
- 117
0
votes
0
answers
22
views
What is the effect of adding variables to an analysis on type 1 and type 2 error?
I am planning a study where we have a low number of observations. We know we need to control for at least two variables, but other variables also exist that we can control for.
It seems to me that ...
- 139
12
votes
2
answers
244
views
The probability of making a Type S error, and the average amount of magnification (type M error) as a function of power
When you filter on only viewing results with p<alpha, some of those "statistically significant" results go in the wrong direction. This is called a Type S error ("S" for sign). ...
- 19.6k
0
votes
0
answers
28
views
Difference in results between anova and Anova (car)
I have a question about using the anova function and Anova from the car package. I know there is a question about Type 1, 2 and 3, but the question is this: I am doing an analysis where I want to ...
1
vote
0
answers
202
views
How to correct for multiple testing in linear mixed models?
I want to correct for alpha error accumulation for my 10 crossed linear mixed models. The random parameters are specified as follows: random intercepts for subject and stimulus and a random slope for ...
- 11
2
votes
1
answer
61
views
Bonferroni correction: Whats exactly is meant by "multiple tests"? [duplicate]
The Bonferroni correction seems to be quite controversial. But I read again and again that it should be used for multiple tests. But what exactly are multiple tests? If I have three different data ...
- 89
0
votes
0
answers
22
views
How to apply Bayesian statistics to this (general) Quality Control problem
This more an academic question than practical. I like that ;)
I have been thoroughly trained in frequentist statistics. Zealously, actually - to the point that no one even told me it’s called “...
- 76
0
votes
1
answer
176
views
post-hoc analysis for interaction terms in logistic regression: emtrends(), type I error
I built a logistic regression, including the followed variables. And I tried to compare effects of temp among group and habitat using emtrends() in ...
1
vote
1
answer
109
views
Visualizing how the 2 sample Z test works
In my stats class, we are talking about the 2 proportion $Z$-test, which compares two sample proportions. The test statistic is given as:
$$
Z = \frac{\hat{p_1}-\hat{p_2}}{\sqrt{\hat{p_c}(1-\hat{p_c})...
- 135
0
votes
0
answers
48
views
Type 1 and Type 2 error with a decision rule with a discrete distributions
Consider the discrete distribution below:
X
0
1
2
3
4
5
6
$p_0$
$a_0$
$a_1$
$a_2$
$a_3$
$a_4$
$a_5$
$a_6$
$p_1$
$b_0$
$b_1$
$b_2$
$b_3$
$b_4$
$b_5$
$b_6$
Suppose $H_0: p_0$ is the correct ...
- 205
0
votes
0
answers
44
views
Type-2 errors in the likelihood-ratio/G-test?
I am quite new to hypothesis testing, and I am currently trying to familiarize myself with the G-test for independence. In my research, I came up with a question for which I could not fit an answer ...
- 545
2
votes
2
answers
67
views
In a muddle with Paired Samples t-tests on pre vs during vs post
I have used paired samples t-tests to compare 3x time-point measurements, A/ B/ C.
A and B t(102)=4.988 p=.000
A and C t(102)=4.939 p=.000
B and C t(102)=.346 p=.730
A, B, and C were all significant ...
1
vote
0
answers
39
views
Need help with Type 1 error and large number of chi squared results
I have a received comment for my research that addresses concern on the 'volume of null hypothesis test' being performed throughout my study and if any corrections are being done to account for 'an ...
- 11
2
votes
2
answers
70
views
What does it mean for tests to be independent?
When reading about cumulation if type-1 Error, the sentence "for independent statistical tests" occures alot, now I was wondering what this is actually means.
Since tests are also random ...
- 163
0
votes
0
answers
45
views
how to deal with the accumulation of the alpha error?
I have six qualitative ( ordinal) dependent variables (psychological distress), all of which have 4 categories (from 1 to 4). and 14 independent demographic variables (e.g gender, level of education, ...
- 68
0
votes
1
answer
73
views
Questions regarding power of test and type II error
I´m preparing for a lecture in decision theory and I´m a little bit confused by the notation used by my prof.
On the first slide under remark 3.2 point v) its written, that $\beta(\varphi)$ is equal ...
1
vote
1
answer
86
views
What is the distribution and standard deviation of the number of Type I errors in hypothesis testing?
My question is from the book "An introduction to statistical learning" (2nd edition), chapter 13, exercise 2.
Suppose that we test m hypotheses, and control the Type I error for each ...
- 85
0
votes
0
answers
25
views
What does look back mean observational studies?
Consider I am doing observational study either case control or cohort in either of the following cases.
Case. 1 Suppose I found enrollment/rate of population with disease is too low where disease ...
- 1,365
1
vote
0
answers
42
views
In linear regression, does the t-test of significance satisfy $\sup_{\beta\in\Theta_0}Pr(\textrm{Reject}\ H_0: \beta_j=0)=\alpha$ asymptotically?
Suppose I have a linear regression $Y=\beta_0+\beta_1X_1+\beta_2X_2+U\equiv X'\beta+U$, where $E(U|X)=0$. After we have found the least squares estimator of $\beta\equiv[\beta_0,\beta_1,\beta_2]$ as $\...
- 2,526
0
votes
0
answers
35
views
Why should we specify statistical analysis a priori? [duplicate]
I understand that multiple post hoc comparisons increase the likelihood of finding spurious associations, but I'm looking for an explanation of how exactly Type 1 and Type 2 errors are affected by not ...
- 1
0
votes
0
answers
20
views
Correcting for multiple comparisons repeated-measures design
I have a question regarding correcting for multiple comparisons in my design; participants were presented with 2 vignettes each describing a person, and for each vignette each participant had to ...
0
votes
0
answers
30
views
Methods to control for false omission rate?
What methods are available to control for false omission rate in multiple comparisons, similar to the way Benjamini-Hochberg controls for false discovery rate?
The goal is to filter out obvious ...
- 253
0
votes
0
answers
99
views
Calculating Type 2 Error
The question given is:
When performing a test regarding the population mean, $\mu$, we decide not to reject $H_0$ when $\bar{X} > 3.$ It is known that $n = 50.$ If the true value for $\mu$ is $\mu ...
1
vote
1
answer
222
views
Equivalent to Dunnett's test for binomial endpoint?
I have a trial comparing 2 treatments vs a control group. Typically one would use Dunnett's test for this situation, but the outcome measure is binomial. It seems strange to me but I've looked for ...
- 549
4
votes
1
answer
264
views
History of terms type 1 error and type 2 error?
The terms "type I" (or "alpha) and "type II" (or "beta) error, to denote false positive and false negative, are often used. What is the history of those terms?
- 41
2
votes
1
answer
99
views
Does p-hacking affect the Type II (False Negative) error rate?
I know that p-hacking (also know as data dredging) increases the Type I error rate as exemplified by this XKCD example.
My question is whether it influences the Type II error rate. I'm not sure ...
- 123
0
votes
0
answers
29
views
$H_2$, its might and Type III error
I've recently found myself wondering about the following thing in hypothesis testing:
We can test some null hypothesis $H_0$ and reject it based on its p-value. This is akin to accepting (shreek!!!) ...
- 261
2
votes
0
answers
261
views
Are Type I and Type II errors in multiclass problem appear to be the same?
As far as I know from binary classification
FP error is a type 1 error
FN error is a type 2 error
I have this confusion matrix generated:
And here I found how to read this confusion matrix:
As you ...
- 121
2
votes
1
answer
250
views
Is it necessary to correct p-values for multiple comparisons in multivariate regression?
In this suspiciously brief comment on another article, the author suggests that analysts should only correct for multiple comparisons in univariate regression when the predictor has more than two ...
- 2,019
2
votes
1
answer
84
views
Should one examine the cross-correlation plot to rule out performing Granger causality test?
Should one look at cross-correlation plot before performing Granger causality test to avoid type I errors?
If we can't find any dependence between two series from the cross-correlation plot, then ...
user318514
3
votes
1
answer
103
views
Two confusion matrices
In a machine learning context, I am working on a binary classification problem. There is a source of truth $T$ for labels, and a labeling process $A$ which is not perfect and makes errors compared to $...
- 1,649
2
votes
0
answers
499
views
For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?
This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size?
In the referenced post, we see that the bootstrap approach does not control type-1 ...
- 21
0
votes
2
answers
273
views
Is confounding a source of Type I errors?
I would be interested to explore the potential sources of Type I errors in randomised controlled trials. A key question in this respect is whether confounding can be understood as one such source.
- 31
2
votes
1
answer
320
views
Is my way of deriving a statistical test from Hoeffding's inequality correct?
I'm trying to deduce from samples of observations from two finite sets of random variables $X_{1}, ..., X_{n}$ and $Y_{1}, ..., Y_{m}$ that the expected values of the average of those random variables ...
- 155
0
votes
0
answers
46
views
Testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$: Find $c$ in rejection region so that probability of a type I error is $\alpha$
My problem is as follows:
Let $X_1,\ldots,X_n$ be a random sample from a $N(\theta,1)$ distribution, and consider testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$. Let $\Omega = \{\mathbf{X}: ...
- 121
2
votes
1
answer
182
views
How to simulate type I error for random-effects model?
Building upon this post How to simulate type I error and type II error. I would like to simulate type I error for a random-effects model I generated. The statistic of interest is standard deviations ...
- 211
0
votes
1
answer
271
views
simple vs simple hypothesis, finding sample size given type 1 and 2 error probabilities
The problem is:
It is decided to investigate the hypotheses H0 : μ = €678 and H1 : μ = €720.
Assuming that the prices are normally distributed with standard deviation €100, what sample size must be ...
2
votes
0
answers
71
views
Measurement error and type I. and type II. error [closed]
Is there a relationship between measurement error (1.) both systematic and random, (2.) systematic only and (3.) random only in terms of tendency of results of statistical tests to be biased towards ...
- 111
2
votes
1
answer
43
views
Simple question on probability computation for multiple comparisons
In class, we illustrated the multiple comparisons problem through a simple example (no, not the xkcd jelly beans comic). Essentially, with $N$ independent tests each at level $\alpha$, we said the ...
- 533
0
votes
0
answers
205
views
Question about power of FDR and FWE
I understand that false discovery rate (FDR) is $\leq$ familywise error rate (FWE). I have read this means that controlling FDR is hence "more powerful" than controlling FWE. What does ...
- 533
3
votes
1
answer
95
views
Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?
I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
- 533
2
votes
1
answer
29
views
How many tests can one do in a medical publication?
Suppose we are in the following setting. There are 10 covariates and 1 response along with some other data derived from those covariates.
One can do regression of 10 covariates to 1 response to test ...
- 1,365
0
votes
1
answer
87
views
Type II error for survival analysis
I am a beginner and I have been looking all over the internet and books to find an answer to the folowing question (but haven't been able to find the answer):
I have observatoinal data and would like ...
- 149
0
votes
0
answers
63
views
Confidence inerval for Empirical Type I error simulation alpha = 0.05
I'm doing a Type I error simulation(10,000 times) at alpha = 0.05. I wonder how can I calculate a confidence interval for the simulated restuls? At a 95% confidence interval, I suppose I can use xbar +...
0
votes
0
answers
30
views
Relationship between Type 1 error and Type 2 error
Given that Type I error is 0.01, what is the highest value possible for Type II error?
- 1
0
votes
0
answers
34
views
Testing pre-conditions before testing a hypothesis
I prefer the Bayesian approach to hypothesis testing myself, and when I look at how traditional null hypothesis testing is taught, I see a problem that is never talked about.
For example:
While ...
- 23