Questions tagged [type-i-and-ii-errors]

Type I: rejecting the null hypothesis when it is true. Type II: not rejecting the null hypothesis when the alternative is true.

Filter by
Sorted by
Tagged with
1 vote
0 answers
25 views

For small sample sizes, is jackknife superior at controlling Type-I error compared to bootstrap?

This question is motivated by the post here: Can bootstrap be seen as a "cure" for the small sample size? In the referenced post, we see that the bootstrap approach does not control type-1 ...
user avatar
  • 11
0 votes
2 answers
38 views

Is confounding a source of Type I errors?

I would be interested to explore the potential sources of Type I errors in randomised controlled trials. A key question in this respect is whether confounding can be understood as one such source.
user avatar
  • 31
0 votes
0 answers
21 views

Find type I and type II errors for t-test using p-value analysis

Say I have some social network's logs, containing $\textit{views}$ and $\textit{likes}$ data (i.e. those are vectors). We would like to make some investigation of accuracy and power of the tests. We ...
user avatar
  • 201
0 votes
0 answers
6 views

How Is Kulldorff spatial scan statistic protected from multiple testing bias?

I recently discovered Kulldorff spatial scan statistic which is used to identify disease clusters in a Poisson or Binomial process: https://sci-hub.se/https://doi.org/10.1080/03610929708831995. It ...
user avatar
  • 2,313
0 votes
0 answers
14 views

Verifying Holm correction line on FWER graph

I'm teaching a class about family-wise error rate, and have created a graph to illustrate how, when we increase the # of comparisons, the rate of type I errors increases rapidly (Family-Wise Error ...
user avatar
  • 132
0 votes
0 answers
6 views

How can one control for FDR if we do not know the prevalence?

I am trying to understand the concept of FDR, and particularly "controlling" the FDR in multiple comparison testing. I undesratdn, and have used many times, the concept of α (type 1 error ...
user avatar
2 votes
1 answer
57 views

Is my way of deriving a statistical test from Hoeffding's inequality correct?

I'm trying to deduce from samples of observations from two finite sets of random variables $X_{1}, ..., X_{n}$ and $Y_{1}, ..., Y_{m}$ that the expected values of the average of those random variables ...
user avatar
  • 155
0 votes
0 answers
31 views

Testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$: Find $c$ in rejection region so that probability of a type I error is $\alpha$

My problem is as follows: Let $X_1,\ldots,X_n$ be a random sample from a $N(\theta,1)$ distribution, and consider testing $H_0: \theta \leq 0$ versus $H_1: \theta > 0$. Let $\Omega = \{\mathbf{X}: ...
user avatar
1 vote
1 answer
55 views

How to simulate type I error for random-effects model?

Building upon this post How to simulate type I error and type II error. I would like to simulate type I error for a random-effects model I generated. The statistic of interest is standard deviations ...
user avatar
  • 155
0 votes
1 answer
47 views

simple vs simple hypothesis, finding sample size given type 1 and 2 error probabilities

The problem is: It is decided to investigate the hypotheses H0 : μ = €678 and H1 : μ = €720. Assuming that the prices are normally distributed with standard deviation €100, what sample size must be ...
user avatar
0 votes
0 answers
28 views

Calculate probability of type II error in single hypothesis test

I have $X \sim N(\mu,4)$, and want to test $H_0:\mu=-1$ vs $H_1:\mu=1$. I have found out so far that after applying the Neyman-Pearson lemma, the critical region has a form $$\left\{ \overline{X_n}>...
user avatar
  • 23
2 votes
0 answers
47 views

Measurement error and type I. and type II. error

Is there a relationship between measurement error (1.) both systematic and random, (2.) systematic only and (3.) random only in terms of tendency of results of statistical tests to be biased towards ...
user avatar
1 vote
1 answer
35 views

Simple question on probability computation for multiple comparisons

In class, we illustrated the multiple comparisons problem through a simple example (no, not the xkcd jelly beans comic). Essentially, with $N$ independent tests each at level $\alpha$, we said the ...
user avatar
0 votes
0 answers
37 views

Question about power of FDR and FWE

I understand that false discovery rate (FDR) is $\leq$ familywise error rate (FWE). I have read this means that controlling FDR is hence "more powerful" than controlling FWE. What does ...
user avatar
3 votes
1 answer
48 views

Does a procedure controlling FDR at level $\alpha$ always reject at least as much as a procedure controlling FWE at $\alpha$?

I understand the false discovery rate (FDR) is weakly less than the familywise error rate (FWE), and FDR is thus a less stringent way to control for type 1 errors. However, will a procedure that ...
user avatar
2 votes
1 answer
22 views

How many tests can one do in a medical publication?

Suppose we are in the following setting. There are 10 covariates and 1 response along with some other data derived from those covariates. One can do regression of 10 covariates to 1 response to test ...
user avatar
  • 839
0 votes
1 answer
26 views

Type II error for survival analysis

I am a beginner and I have been looking all over the internet and books to find an answer to the folowing question (but haven't been able to find the answer): I have observatoinal data and would like ...
user avatar
1 vote
0 answers
26 views

Why do we have a Neyman-Pearson lemma for type one error but not an analogous one for precision and recall of a classifier?

We fix alpha to .05 in statistical testing. We never fix the precision or recall of a test. There's no decision theory lemma that guarantees we can find a decision region with fixed precision or ...
user avatar
  • 224
0 votes
0 answers
17 views

Confidence inerval for Empirical Type I error simulation alpha = 0.05

I'm doing a Type I error simulation(10,000 times) at alpha = 0.05. I wonder how can I calculate a confidence interval for the simulated restuls? At a 95% confidence interval, I suppose I can use xbar +...
user avatar
0 votes
0 answers
25 views

Is this a multiple comparison problem?

I am planning to conduct an experimental study (2 groups, 1 measure) of which there will be three possible results. Either there will be an increase in the measure between the experimental and control ...
user avatar
  • 21
0 votes
0 answers
23 views

Relationship between Type 1 error and Type 2 error

Given that Type I error is 0.01, what is the highest value possible for Type II error?
user avatar
  • 1
0 votes
0 answers
22 views

Testing pre-conditions before testing a hypothesis

I prefer the Bayesian approach to hypothesis testing myself, and when I look at how traditional null hypothesis testing is taught, I see a problem that is never talked about. For example: While ...
user avatar
  • 23
0 votes
1 answer
159 views

What is a two-sided type 1 error rate?

This is probably a simple question but I was watching an online video about a scientific study. For the study, they mentioned that they used a two-tailed Type 1 error rate of 0.05. I know that a Type ...
user avatar
  • 173
3 votes
0 answers
107 views

Why the p value of the minimum P value is powerful?

I am reading a genetics paper: A Powerful and Adaptive Association Test for Rare Variants. In this paper, there are several hypothesis tests, $H_1, \dotsc, H_r$, and their corresponding p-values, $p_1,...
user avatar
  • 41
2 votes
2 answers
141 views

Can all possible pairs of comparison be planned comparisons in ANOVA?

Let's say we have three intervention groups (condition A, condition B, and condition C/control) to be analyzed in ANOVA, and we are theoretically interested in the difference between each pair of all ...
user avatar
  • 23
2 votes
1 answer
56 views

Controlling false positive rate and false negative rate

My notes define the false positive rate as $$\dfrac{\text{false positives}}{\text{true negatives} + \text{false positives}} = \dfrac{\text{false positives}}{\text{total negatives}}$$ and the false ...
user avatar
  • 1,096
2 votes
1 answer
406 views

(Solved) Hypothesis Testing: Probability $\beta$ of Type II Error in Proportions

$\newcommand{\szdp}[1]{\!\left(#1\right)}$ Background Question: A manufacturer claimed that at least $20\%$ of the public preferred his product. A sample of $100$ persons is taken to check his claim. ...
user avatar
0 votes
0 answers
37 views

What is social science and discussion about Type II error preference?

Today, my senior lecturer in economics and finance class teaching about laws impact companies' operation told us one sentence "Also keep in mind that in (at least social) science, we care (try to ...
user avatar
1 vote
0 answers
19 views

Type II error: statistically nonsignificant but can be actually similar

I am analysing a rare disease, meaning that a low sample size is inevitable (n = 50 for one group in my case). And I came to a question described as follows. Could we somehow differentiate the ...
user avatar
3 votes
2 answers
66 views

Can you actually prove if the null hypothesis is true or a type II error has occurred?

Can you actually prove if the null hypothesis is true or a type II error has occurred? I'm new to statistics (sorry in advance!)
user avatar
  • 41
0 votes
1 answer
67 views

How do I correct for type I error in a mediation when using Hayes' PROCESS in SPSS?

I have performed three mediation analyses using PROCESS in SPSS, each with a different outcome/dependent variable. To prevent a type I error, I would like to perform a correction for this, but seeing ...
user avatar
  • 1
3 votes
1 answer
174 views

Two sample t test with equal variances hypothesis

Given a trial to test a new drug. There are 20 volunteers with some disease. The volunteers are split into two groups (10 volunteers in each) randomly: the treatment group and the control group. ...
user avatar
  • 35
0 votes
0 answers
60 views

When does a proxy/substitute variable distinguish between groups as well as the original variable?

I’m trying to determine when a proxy variable can be substituted for a real variable for the purposes of distinguishing two groups. To make this more concrete, suppose I want to determine, generally, ...
user avatar
1 vote
1 answer
42 views

Why reversing a continuous measure leads to a non-significant correlation coefficient

Background: I'm analyzing correlation between two behavioural types (boldness and aggression). Boldness values are continuous (range: 2 to 1195) and it's unit of measurement is in seconds (latencies). ...
user avatar
  • 21
0 votes
1 answer
51 views

Hypothesis testing, type I and II error

A sample $X_1, X_2, X_3, X_4$ comes from a normal distribution $N(m, 2^2)$. To verify $H_0: m=4$ against $H_1: m = 1$ one uses a test with a critical region: $K = \{4X_1 − 2X_2 − 2X_3 + X_4 < −2\}$....
user avatar
  • 15
0 votes
0 answers
38 views

Type I error level vs. sample size

From Jay and Devore's Modern Mathematical Statistics, Chapter 9, problem 61 c In most situations, would it be reasonable to use a level .01 test in conjunction with a sample size of 40,000? Why or ...
user avatar
1 vote
2 answers
87 views

Estimating minimal sample size for two sample test (comparing means)

If I want to perform a hypothesis test to compare the means of 2 equally big samples from normal distributions with equal variances, how would I compute the minimum required sample size for each ...
user avatar
  • 47
1 vote
0 answers
54 views

Why is the probability of a type 1 error, $\alpha$, the significance level?

I've seen this stated Why are p-values uniformly distributed under the null hypothesis? and https://support.minitab.com/en-us/minitab-express/1/help-and-how-to/basic-statistics/inference/supporting-...
user avatar
2 votes
0 answers
35 views

Why a false positive is called type I error and a flase negative a type II error? [duplicate]

I am trying to understand what are the historical reasons behind the choice of the term Type I and Type II error. I think is much more intuitive to use false positive and false negative. These two ...
user avatar
  • 291
1 vote
1 answer
180 views

linear mixed effects models: should I aggregate repeated trials?

A colleague of mine asked me to analyse some data from a movement experiment. Participants were asked to turn their head three times to the left (toL), three times to the right (toR), with both the ...
user avatar
  • 41
0 votes
1 answer
96 views

Correction for multiple comparisons in a linear mixed effects model with dummy coded categorical predictor

I'm analysing the data from a multi-group randomised controlled trial. Participants were assigned to one of five intervention groups, or a control group. They were tested twice, pre (T1) and post-...
user avatar
3 votes
0 answers
2k views

Cross validation and type I, type II error

In the context of building a predictive model, I understand that cross validation (such as K-Fold) is a technique to find the optimal hyper-parameters in reducing bias and variance somewhat. Recently, ...
user avatar
  • 193
0 votes
1 answer
30 views

What would be the most appropriate test to compare two groups in the three parts of the data?

I have a response dataset for twenty-five questions, which measures three grammar topics. The participants are two different English proficiency groups of intermediate and advanced. So, I want to ...
user avatar
0 votes
1 answer
81 views

Definition of a confidence interval

In one exercise of a mathematical statistics lecture, the teaching assistant wrote the following : A confidence interval for a given parameter $\theta$ is an interval [A,B] at level $\alpha$ such ...
user avatar
0 votes
0 answers
50 views

Can the null hypothesis be **more** true than the alternative hypothesis or vice versa? [duplicate]

As the title says, my question is can the null hypothesis ($H_0$) be more true than the alternative hypothesis ($H_1$) or vice versa? For example, if the statistical power is really high (extremely ...
user avatar
0 votes
0 answers
62 views

Choice of critical/rejection region and relation of power function and type I/II error

A hypothesis test is, fundamentally, defined by a rejection region $R$(a set of the sample $\mathbf{X}$); not by statics such as LRT and MLE etc, but by a range (Must be so, see the following comment.)...
user avatar
0 votes
0 answers
68 views

Type I Type II error with power function as y-axis and mu(alternative) as x-axis

I stuck in the following graph for 2 days. I am confused because I ask myself the following two questions but did not get solved by myself. Q1: If Null (H0) is larger or equal to 5, then why when mu(...
user avatar
1 vote
2 answers
200 views

Orthogonal contrasts and the family-wise error rate

I recently read about contrasts and how to code them. Contrasts can be orthogonal which means that they are independent and the statistical tests for them are not correlated. Normally, when doing ...
user avatar
0 votes
0 answers
30 views

Type II error relevant limitation when Kaplan-Meier curves are identical?

Let's suppose that we have a made-up data including two diseases, and one of them is very rare (impossible to get a large sample size). Kaplan-Meier curves of the two diseases are almost identical, ...
user avatar
  • 1,533
0 votes
0 answers
23 views

Permuting outcome variable to estimate null likelihood ratio distribution for Poisson GLM

I fit a Poisson GLM using a set of candidate explanatory variables ($\lambda = e^{\beta_{0} + \beta_{1}x_{1}+\beta_{2}x_{2}+...}$), tested against the global null hypothesis ($\lambda = e^{\beta_{0}}$)...
user avatar

1
2 3 4 5 6