Questions tagged [type-i-and-ii-errors]

Type I: rejecting the null hypothesis when it is true. Type II: not rejecting the null hypothesis when the alternative is true.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

Build three models in one study

I build three models. First model has two mediators, one is not significant, then I removed it from the second model. The second one has one moderator and one mediator. The third one has another ...
  • 1
0 votes
0 answers
10 views

Mitigating Two-Sample Difference Type I Error

I am currently comparing the distributions of two populations, $P_a$ and $P_b$. My access to those two populations comes from filtering1 a broader dataset $D$ into sample sets $D_a$ and $D_b$. ...
4 votes
1 answer
171 views

History of terms type 1 error and type 2 error?

The terms "type I" (or "alpha) and "type II" (or "beta) error, to denote false positive and false negative, are often used. What is the history of those terms?
2 votes
1 answer
53 views

Does p-hacking affect the Type II (False Negative) error rate?

I know that p-hacking (also know as data dredging) increases the Type I error rate as exemplified by this XKCD example. My question is whether it influences the Type II error rate. I'm not sure ...
  • 123
0 votes
0 answers
24 views

$H_2$, its might and Type III error

I've recently found myself wondering about the following thing in hypothesis testing: We can test some null hypothesis $H_0$ and reject it based on its p-value. This is akin to accepting (shreek!!!) ...
1 vote
0 answers
84 views

Are Type I and Type II errors in multiclass problem appear to be the same?

As far as I know from binary classification FP error is a type 1 error FN error is a type 2 error I have this confusion matrix generated: And here I found how to read this confusion matrix: As you ...
  • 111
1 vote
1 answer
52 views

Is it necessary to correct p-values for multiple comparisons in multivariate regression?

In this suspiciously brief comment on another article, the author suggests that analysts should only correct for multiple comparisons in univariate regression when the predictor has more than two ...
  • 1,483
2 votes
1 answer
21 views

Should one examine the cross-correlation plot to rule out performing Granger causality test?

Should one look at cross-correlation plot before performing Granger causality test to avoid type I errors? If we can't find any dependence between two series from the cross-correlation plot, then ...
2 votes
1 answer
68 views

Two confusion matrices

In a machine learning context, I am working on a binary classification problem. There is a source of truth $T$ for labels, and a labeling process $A$ which is not perfect and makes errors compared to $...
  • 1,397
1 vote
0 answers
105 views

For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?

This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size? In the referenced post, we see that the bootstrap approach does not control type-1 ...
  • 11
0 votes
2 answers
60 views

Is confounding a source of Type I errors?

I would be interested to explore the potential sources of Type I errors in randomised controlled trials. A key question in this respect is whether confounding can be understood as one such source.
  • 31
0 votes
0 answers
24 views

Find type I and type II errors for t-test using p-value analysis

Say I have some social network's logs, containing $\textit{views}$ and $\textit{likes}$ data (i.e. those are vectors). We would like to make some investigation of accuracy and power of the tests. We ...
  • 201
0 votes
0 answers
7 views

How Is Kulldorff spatial scan statistic protected from multiple testing bias?

I recently discovered Kulldorff spatial scan statistic which is used to identify disease clusters in a Poisson or Binomial process: https://sci-hub.se/https://doi.org/10.1080/03610929708831995. It ...
  • 2,343
0 votes
0 answers
22 views

Verifying Holm correction line on FWER graph

I'm teaching a class about family-wise error rate, and have created a graph to illustrate how, when we increase the # of comparisons, the rate of type I errors increases rapidly (Family-Wise Error ...
  • 132
0 votes
0 answers
6 views

How can one control for FDR if we do not know the prevalence?

I am trying to understand the concept of FDR, and particularly "controlling" the FDR in multiple comparison testing. I undesratdn, and have used many times, the concept of α (type 1 error ...
2 votes
1 answer
109 views

Is my way of deriving a statistical test from Hoeffding's inequality correct?

I'm trying to deduce from samples of observations from two finite sets of random variables $X_{1}, ..., X_{n}$ and $Y_{1}, ..., Y_{m}$ that the expected values of the average of those random variables ...
  • 155
0 votes
0 answers
36 views

Testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$: Find $c$ in rejection region so that probability of a type I error is $\alpha$

My problem is as follows: Let $X_1,\ldots,X_n$ be a random sample from a $N(\theta,1)$ distribution, and consider testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$. Let $\Omega = \{\mathbf{X}: ...
1 vote
1 answer
77 views

How to simulate type I error for random-effects model?

Building upon this post How to simulate type I error and type II error. I would like to simulate type I error for a random-effects model I generated. The statistic of interest is standard deviations ...
  • 155
0 votes
1 answer
70 views

simple vs simple hypothesis, finding sample size given type 1 and 2 error probabilities

The problem is: It is decided to investigate the hypotheses H0 : μ = €678 and H1 : μ = €720. Assuming that the prices are normally distributed with standard deviation €100, what sample size must be ...
0 votes
0 answers
30 views

Calculate probability of type II error in single hypothesis test

I have $X \sim N(\mu,4)$, and want to test $H_0:\mu=-1$ vs $H_1:\mu=1$. I have found out so far that after applying the Neyman-Pearson lemma, the critical region has a form $$\left\{ \overline{X_n}>...
  • 35
2 votes
0 answers
53 views

Measurement error and type I. and type II. error [closed]

Is there a relationship between measurement error (1.) both systematic and random, (2.) systematic only and (3.) random only in terms of tendency of results of statistical tests to be biased towards ...
1 vote
1 answer
35 views

Simple question on probability computation for multiple comparisons

In class, we illustrated the multiple comparisons problem through a simple example (no, not the xkcd jelly beans comic). Essentially, with $N$ independent tests each at level $\alpha$, we said the ...
0 votes
0 answers
83 views

Question about power of FDR and FWE

I understand that false discovery rate (FDR) is $\leq$ familywise error rate (FWE). I have read this means that controlling FDR is hence "more powerful" than controlling FWE. What does ...
3 votes
1 answer
56 views

Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?

I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
2 votes
1 answer
23 views

How many tests can one do in a medical publication?

Suppose we are in the following setting. There are 10 covariates and 1 response along with some other data derived from those covariates. One can do regression of 10 covariates to 1 response to test ...
  • 1,159
0 votes
1 answer
35 views

Type II error for survival analysis

I am a beginner and I have been looking all over the internet and books to find an answer to the folowing question (but haven't been able to find the answer): I have observatoinal data and would like ...
  • 139
1 vote
0 answers
29 views

Why do we have a Neyman-Pearson lemma for type one error but not an analogous one for precision and recall of a classifier?

We fix alpha to .05 in statistical testing. We never fix the precision or recall of a test. There's no decision theory lemma that guarantees we can find a decision region with fixed precision or ...
0 votes
0 answers
25 views

Confidence inerval for Empirical Type I error simulation alpha = 0.05

I'm doing a Type I error simulation(10,000 times) at alpha = 0.05. I wonder how can I calculate a confidence interval for the simulated restuls? At a 95% confidence interval, I suppose I can use xbar +...
0 votes
0 answers
25 views

Is this a multiple comparison problem?

I am planning to conduct an experimental study (2 groups, 1 measure) of which there will be three possible results. Either there will be an increase in the measure between the experimental and control ...
  • 21
0 votes
0 answers
24 views

Relationship between Type 1 error and Type 2 error

Given that Type I error is 0.01, what is the highest value possible for Type II error?
  • 1
0 votes
0 answers
24 views

Testing pre-conditions before testing a hypothesis

I prefer the Bayesian approach to hypothesis testing myself, and when I look at how traditional null hypothesis testing is taught, I see a problem that is never talked about. For example: While ...
  • 23
0 votes
1 answer
436 views

What is a two-sided type 1 error rate?

This is probably a simple question but I was watching an online video about a scientific study. For the study, they mentioned that they used a two-tailed Type 1 error rate of 0.05. I know that a Type ...
  • 223
3 votes
0 answers
112 views

Why the p value of the minimum P value is powerful?

I am reading a genetics paper: A Powerful and Adaptive Association Test for Rare Variants. In this paper, there are several hypothesis tests, $H_1, \dotsc, H_r$, and their corresponding p-values, $p_1,...
  • 41
3 votes
2 answers
241 views

Can all possible pairs of comparison be planned comparisons in ANOVA?

Let's say we have three intervention groups (condition A, condition B, and condition C/control) to be analyzed in ANOVA, and we are theoretically interested in the difference between each pair of all ...
  • 33
2 votes
1 answer
81 views

Controlling false positive rate and false negative rate

My notes define the false positive rate as $$\dfrac{\text{false positives}}{\text{true negatives} + \text{false positives}} = \dfrac{\text{false positives}}{\text{total negatives}}$$ and the false ...
  • 1,186
2 votes
1 answer
636 views

(Solved) Hypothesis Testing: Probability $\beta$ of Type II Error in Proportions

$\newcommand{\szdp}[1]{\!\left(#1\right)}$ Background Question: A manufacturer claimed that at least $20\%$ of the public preferred his product. A sample of $100$ persons is taken to check his claim. ...
0 votes
0 answers
38 views

What is social science and discussion about Type II error preference?

Today, my senior lecturer in economics and finance class teaching about laws impact companies' operation told us one sentence "Also keep in mind that in (at least social) science, we care (try to ...
1 vote
0 answers
19 views

Type II error: statistically nonsignificant but can be actually similar

I am analysing a rare disease, meaning that a low sample size is inevitable (n = 50 for one group in my case). And I came to a question described as follows. Could we somehow differentiate the ...
3 votes
2 answers
70 views

Can you actually prove if the null hypothesis is true or a type II error has occurred?

Can you actually prove if the null hypothesis is true or a type II error has occurred? I'm new to statistics (sorry in advance!)
  • 41
0 votes
1 answer
88 views

How do I correct for type I error in a mediation when using Hayes' PROCESS in SPSS?

I have performed three mediation analyses using PROCESS in SPSS, each with a different outcome/dependent variable. To prevent a type I error, I would like to perform a correction for this, but seeing ...
  • 1
3 votes
1 answer
252 views

Two sample t test with equal variances hypothesis

Given a trial to test a new drug. There are 20 volunteers with some disease. The volunteers are split into two groups (10 volunteers in each) randomly: the treatment group and the control group. ...
  • 135
1 vote
1 answer
44 views

Why reversing a continuous measure leads to a non-significant correlation coefficient

Background: I'm analyzing correlation between two behavioural types (boldness and aggression). Boldness values are continuous (range: 2 to 1195) and it's unit of measurement is in seconds (latencies). ...
  • 21
0 votes
1 answer
54 views

Hypothesis testing, type I and II error

A sample $X_1, X_2, X_3, X_4$ comes from a normal distribution $N(m, 2^2)$. To verify $H_0: m=4$ against $H_1: m = 1$ one uses a test with a critical region: $K = \{4X_1 − 2X_2 − 2X_3 + X_4 < −2\}$....
  • 15
0 votes
0 answers
43 views

Type I error level vs. sample size

From Jay and Devore's Modern Mathematical Statistics, Chapter 9, problem 61 c In most situations, would it be reasonable to use a level .01 test in conjunction with a sample size of 40,000? Why or ...
1 vote
2 answers
132 views

Estimating minimal sample size for two sample test (comparing means)

If I want to perform a hypothesis test to compare the means of 2 equally big samples from normal distributions with equal variances, how would I compute the minimum required sample size for each ...
  • 47
1 vote
0 answers
92 views

Why is the probability of a type 1 error, $\alpha$, the significance level?

I've seen this stated Why are p-values uniformly distributed under the null hypothesis? and https://support.minitab.com/en-us/minitab-express/1/help-and-how-to/basic-statistics/inference/supporting-...
2 votes
0 answers
35 views

Why a false positive is called type I error and a flase negative a type II error? [duplicate]

I am trying to understand what are the historical reasons behind the choice of the term Type I and Type II error. I think is much more intuitive to use false positive and false negative. These two ...
  • 291
1 vote
1 answer
252 views

linear mixed effects models: should I aggregate repeated trials?

A colleague of mine asked me to analyse some data from a movement experiment. Participants were asked to turn their head three times to the left (toL), three times to the right (toR), with both the ...
  • 41
0 votes
1 answer
211 views

Correction for multiple comparisons in a linear mixed effects model with dummy coded categorical predictor

I'm analysing the data from a multi-group randomised controlled trial. Participants were assigned to one of five intervention groups, or a control group. They were tested twice, pre (T1) and post-...
3 votes
0 answers
3k views

Cross validation and type I, type II error

In the context of building a predictive model, I understand that cross validation (such as K-Fold) is a technique to find the optimal hyper-parameters in reducing bias and variance somewhat. Recently, ...
  • 193

1
2 3 4 5 6