Questions tagged [type-i-and-ii-errors]

Type I: rejecting the null hypothesis when it is true. Type II: not rejecting the null hypothesis when the alternative is true.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

standard name for area under type 1 vs type 2 error curve?

we already have precision-recall, and roc curves, why aren't type 1-type 2 error curves more popular? Is there a standard name for the area under such plots?
0
votes
0answers
66 views

Should bootstrapping and collecting sample means from a series of binomial distributions result in standard normal?

I’m trying to better understand several statistical concepts (bootstrapping, central limit theorem, and confidence intervals) by applying them to a binomial distribution (you can think of it as a coin ...
5
votes
2answers
101 views

Is it possible to simultaneously call multiple regression coefficients significant?

Okay, this is a thought experiment: Suppose you have a dataset with 40 covariates. Suppose the data has every nice theoretical property you could want: randomly sampled, variables not correlated, etc....
0
votes
0answers
7 views

Errors if running multiple logistic models with same predictors but different dependent variables?

I have a number of predictors and dependent variables that I intend to run with logistic regression. The DVs are binary and IVs are binary and ordinal. My question is, if I run multiple models with ...
1
vote
1answer
41 views

“Optional stopping” to *reduce* type-I-error

I ran an experiment to test for some effects (which I indeed see evidence for), but in the analysis, also found a surprising effect that I could find no theoretical justification for. Specifically, ...
4
votes
2answers
75 views

Is it possible to find out the probability of a type 2 error in 2 sample t test?

I was wondering if it's possible to see what is the probability of getting a type 2 error in my investigation into whether there is a difference in length of time between cells in anaphase and ...
1
vote
0answers
16 views

Multilevel models and correcting for multiple comparisons

I am conducting a study where there are three different DVs. For each DV I will be conducting a multilevel model with 2 predictors and will be interested in both main effects and interactions between ...
1
vote
0answers
20 views

Interim analysis and alpha spending in group sequence design of clinical trials

In group sequential design, does the type of interim analysis affect the alpha spending for each look? I tested in ADDPLAN, with two interim analyses, specifying "stopping for futility" or not does ...
-1
votes
1answer
29 views

Alpha and Hypothesis Testing [closed]

(The type I error, alpha) Which one of the statements about the alpha level is correct? a) The alpha level is the predefined probability of making a mistake when the null hypothesis is false. b) The ...
-1
votes
1answer
30 views

What is the difference between type I error and type II error in default or non-default case?

What is the difference between type I error and type II error in forecasting default of a company? For example, I can set null hypothesis to be "The company will default next year", or I can set it to ...
3
votes
1answer
68 views

P-values in multiple comparison t-test

I'm a biologist trying to teach my college class about the issue of multiple comparisons with t-tests. I've explained that the alpha value must be "combined" so to speak from each of the related ...
0
votes
0answers
12 views

Deriving the experiment-wise error rate of testing all hypotheses at the same time

$\textbf{Q}$: Suppose that we are conducting the tests on the hypotheses $H_i , i = 1, \dots ,k$. For each hypothesis, a test statistic $T_i$ is formed, where each $T_i, i = 1, \dots , k$ are ...
0
votes
0answers
29 views

Nonlinear Wald Test

Say we have a bivariate distribution $~~~~~f(x,y|\theta=\{a,b,t,c\}).$ An event of interest is when ab=c, that is, $~~~~~H_0: g(\theta)=c-ab=0$ versus $H_1: g(\theta)=c-ab \neq 0.$ I attempt to ...
4
votes
1answer
89 views

In what sense does interim monitoring of clinical trials “cost” a Bayesian?

I have read (and will seek a specific reference on the subject) that unlike Frequentist trials, Bayesians can continually monitor data as it accrues. A Frequentist tries to control, and thus ...
5
votes
1answer
129 views

Family-wise Type I error OLS regression

Why is it advised to control the Type I error rate (e.g. Turkey's HSD) when conducting several pairwise comparisons, but not when assessing the significance of several coefficient estimates when ...
2
votes
1answer
57 views

What is allowable deviation from desired significance/coverage?

Consider performing a statistical hypothesis test and wants to achieve significance $\alpha$. What is a good rule of thumb for an allowable deviation from the desired confidence level? Say I want to ...
3
votes
2answers
133 views

Why must a null hypothesis contain equality?

Using the inference of mean as an example, the null and alternative hypothesis could be $$H_0: \mu \le 0 \Leftrightarrow H_1: \mu > 0$$ It is often argued that this makes the calculation of the $...
1
vote
1answer
55 views

Using the $Z$-test instead of the binomial test for small $n$ - effect on power and Type I error

Setup Let $M \sim \text{Bin}(p,n)$. Let us then state the following two hypotheses: $$\begin{align} H_0: p = 0.5 \text{ vs } H_1: p > 0.5 \end{align}$$ To test these hypotheses, I could use the ...
0
votes
0answers
8 views

RNA-seq: probabiliy of at least one type I error

I need to find a mathematical expression for the probability of finding at leat one type I error. I know that in general that probabilty is: P=1-(1-a)^m. With a being the probabily of making a type I ...
1
vote
1answer
65 views

Power of a Poisson hypothesis test

A textbook of mine has online worked solutions, and the answer in the back of the book does not match the answer in the online solutions - they're not even close. My book says that the power of a ...
1
vote
1answer
27 views

Does a more powerful test mechanically lead to more type III errors then?

"Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. Say that a treatment increases some ...
0
votes
0answers
78 views

Uncertainty in Fit Parameters from R^2 and/or Independent Variable Uncertainties

Suppose that I am using a least-squares method to fit experimental data $(x,y)$ to the equation $$y = m x + b.$$ In order to determine $\sigma_m$ and $\sigma_b$, the uncertainty in the parameters $m$ ...
0
votes
1answer
30 views

Calculating type two error without std dev

I have this problem I’m working through that requires me to calculate the type two error but doesn’t supply the std Dev or any way to calculate it. A preschool teacher wants to test to see if more ...
0
votes
1answer
13 views

Tukey Kramer and the reduction of type one error

I hope everyone is doing well. I understand that the Tukey Kramer reduces the chance of getting a type 1 error by using a pooled variance. That pooled variance then gives you a lower standard error ...
1
vote
1answer
44 views

Is a Type III statistical error just a subset of Type I or II error?

I have recently come across the idea of a type III error through discussion with colleagues. The definition they gave me was something along these lines: "Type III error occurs when you correctly ...
0
votes
0answers
91 views

increasing accuracy of binary logistic regression by reducing type II error?

I am to use binary logistic regression predict a deadly disease from 109 cases out of 385 patients. If during the preliminary diagnosis all patients were sent to the expert doctor for secondary ...
1
vote
0answers
27 views

Multiple Testing - FWER

I am confused about Family Wise Error Rate(FWER) when dealing with multiple testing. What would change about the FWER in the case of positively correlated p-values. For example if I perform ...
1
vote
2answers
115 views

proof for the hypothesis testing $\alpha-\beta$ tradeoff

All the statistic's courses state that there is a tradeoff between the significance level ($\alpha$) and the probability of the Type II error ($\beta$). Specifically, for a fixed sample size, one ...
0
votes
0answers
141 views

Type 1 Error correction for multiple comparisons: ANOVA vs multiple regression

After looking into Type 1 error correction protocols for multiple comparisons in ANOVA vs. MR, I found two (conflicting?) messages. The same Type 1 Error corrections that apply to comparisons in ...
0
votes
0answers
25 views

Is it possible to experimentally reproduce the probability of type 1 errors in significance tests?

In some fields, a lot of sample studies feature statistical tests. Oftentimes, box plots of sample distributions are shown. I was wondering whether i could read distributional parameters from these ...
1
vote
0answers
28 views

Sample size formulae

In order to calculate sample size for hypothesis testing, we consider level of significance and power. For level of significance, two sided values ($Z_{\alpha/2}$) are taken into account, but for ...
0
votes
1answer
38 views

Type I error question

I have this question from a previous exam, since it didn't worth much points I suspect it should be something tricky. How to achieve a statistical test where the Type I error risk is $0$? Is it ...
0
votes
1answer
48 views

How to measure risk of a Type 2 error in A/B tests

I'm conducting A/B tests where outcomes are binary (success or failure in control and experiment populations). The assessment of the risk of a Type 1 error is relatively straightforward: given an ...
1
vote
0answers
47 views

Finding the probability of a type II error for a poisson distribution

A poisson distribution is being tested with the hypothesis H0: λ = 1.7 and H1: λ ≠ 1.7 The true value of the mean is 2.3. For the significant level 10% calculate: a )The critical region b) The ...
0
votes
1answer
444 views

Finding the probability of a type II error for a binomial distribution

For a binomial distribution, the hypothesis $H_0: p = 0.2$ and $H_1: p\ne0.2$ are tested at the $10%$ level. $20$ trials are performed and the critical region is $X = 0$ or $X > 7$. Calculate the ...
2
votes
1answer
148 views

Sample size calculation for Kendall's Tau in reproducibility study

Can anyone help with justifying (or rubbishing) a couple of aspects of a sample size calculation for a grant proposal. A pilot study gave significant results for the correlation of a continuous (non-...
0
votes
1answer
56 views

Type-I and Type-II errors example explanation

In the following example, can someone help understand how the value for $k$ (highlighted in yellow) is derived below? How is $z_\alpha$ introduced in there?
0
votes
1answer
113 views

Type I and Type II errors in Hypothesis Testing

I am confused about the last highlighted sentence regarding finding a subset S, for which BOTH Type I and Type II error probabilities are 0. For $P_\theta S$, which is the probability with which we ...
0
votes
0answers
158 views

Empirical Type I error from simulation different from $\alpha$ and changes based on sample size $n$

I am simulating LRT based on Chi-squared random variables. I know that the asymptotic distribution of LRT statistic follows chi-squared distribution with $df=$difference of the degrees of freedom ...
0
votes
1answer
36 views

Probability of Type II error for simple hypothesis

I'm working through Intro to Probability and Statistics... and I'm a little confused about how to calculate the probability of a type II error, particularly as it relates to a simple hypothesis test, ...
3
votes
3answers
1k views

Is there a constraint on the sum of the type-I & type II error probabilities?

Is it true that if $H_0$ and $H_a$ are complementary hypotheses of the Binomial trial, i.e., the negation of $H_0$ is $H_a$ then the type-I error $\alpha$ plus type-II error $\beta$ equals 1? Or is ...
3
votes
3answers
512 views

Does multicollinearity cause type I errors?

Citing Wikipedia on multicollinearity, One of the features of multicollinearity is that the standard errors of the affected coefficients tend to be large. In that case, the test of the hypothesis ...
3
votes
1answer
144 views

Can I perform a Chi Square test comparing the one year mortality of my 160 patients versus the 108000 patients of a cohort in the litterature?

I have exactly zero experience with statistics. I was interested in comparing 1 year mortality of patients $\ge$ 60 years old versus $<$ 60 in a cohort after a specific intervention. I performed ...
2
votes
1answer
108 views

Type I and Type II errors when varying the number of treatments

I have a hypothetical sample of n observations, and I want to test if there are significant differences across multiple treatments. I need to know how the number of treatments interfere in the type I ...
10
votes
2answers
2k views

“Liberal” p-values?

My question is rather semantic. When a method routinely produces high p-values it is called conservative. Would you call the opposite, i.e. a method with a high type-II-error rate liberal?
0
votes
2answers
27 views

Minimize Type II errors in black-box testing

Okay so I have a function in the shape of a black box that I'd like to test. It's characteristics are: It gives either 0, 1 or -1 as output. I can query it easily, producing tens of thousands of ...
21
votes
1answer
1k views

Why the names Type 1, 2 error?

What is the motivation of introducing an additional level of indirection from the descriptive 'false positive' to the integer '1'? Is 'false positive' really too long?
0
votes
1answer
68 views

Calculate type 2 error using one data file

I am a bit confused on how to calculate type 2 error to check whether the sample data I am using is sufficient. I have a data file that I have used to build a machine learning model. This data file ...
0
votes
0answers
98 views

How to get Type II error probability from G*Power

I realize that having standard error of the mean, critical value, sample mean values from a simple a priori power test in the G*Power free download, one could use NORM.DIST in Excel (Argh!) following ...
0
votes
0answers
122 views

What is the smallest value of r that gives a test of size not exceeding 0.05?

I've been given a question that I don't understand. Question: Consider a test which rejects $H_0$ if the number of observed successes is $≥ r$ , where $r$ is some number. What is the smallest value ...

1 2 3 4 5