Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

0
votes
0answers
17 views

Statistical power for AB test

I am planning to run an AB test in Jul-Aug 2019 with 20% size of B group. Before launching, I want to estimate the power in order to make sure I set the correct sample size and make sure the test is ...
1
vote
1answer
50 views

Power of a Poisson hypothesis test

A textbook of mine has online worked solutions, and the answer in the back of the book does not match the answer in the online solutions - they're not even close. My book says that the power of a ...
1
vote
0answers
16 views

simr power simulation testing interaction term for 2 factors

I have a model with two factors: a (4 levels) and b (3 levels). Each participant receives two problems (id as a random effect). I want to estimate sample size based on pilot data to give me ...
1
vote
1answer
18 views

Need help determining sample size for an AB test with two metrics

I need to run an experiment with customer support agents who are rated on two metrics by the customer: average issue resolution rate (proportion of issues resolved) and average star rating (1-5). ...
0
votes
1answer
11 views

Explanation of Advantages of Balanced Designs with Statistical Theory

I wanted to know what the difference between balanced and unbalanced design. Firstly, here are two advantages of balanced designs: ...
1
vote
1answer
23 views

Does a more powerful test mechanically lead to more type III errors then?

"Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. Say that a treatment increases some ...
0
votes
0answers
8 views

Power of Tests for Proportions: One Proportion vs. Two Proportions

I am trying to calculate the required sample size for a trial, where my outcome is proportions (for each subject I am measuring if the subject had an event or not). I tried two ways and got different ...
1
vote
2answers
34 views

Significance of effect size less then estimated by power

Suppose I estimate a sample size needed for a hypothesis test of proportion difference, using a binomial distributional assumption, with power 80% and effect size 25%. Then I perform an A/B ...
1
vote
1answer
30 views

Is Fisher's exact test the most powerful test for 2x2 contingency table?

I don't find the powerfulness of Fisher's exact test on Wikipedia. Are there any results about the power of FET vs other methods on 2x2 contingency tables?
0
votes
0answers
14 views

A question about power for a cohort study

In short, we are planing a cohort study, were we want to investigate the occurrence and the importance of a given event on response of a treatment. We expect the event to occur in about 2% of all the ...
0
votes
0answers
25 views

Plot power of a hypothesis test with $\alpha$

Let's take a simple one-sided hypothesis test (say the two sample t-test). The confidence level we want (probability of rejecting the null when it is true) is defined as $\alpha$ - usually set to 5%. ...
1
vote
1answer
20 views

sample size calculation in 3-arm survival analysis

In a 3-arms clinical trial, with time to event data, either A is superior to C or B is superior to C it will be considered significant. No need to compare A and B. I know how to calculate the sample ...
0
votes
0answers
10 views

How does variation in the effect size affects power?

Effect size The effect size affect the power of a statistical test. We typically summarizes the magnitude of the effect of a variable as a single number (which we call the effect size). To my ...
1
vote
0answers
19 views

How to calculate a sample size for a longitudinal study with repeated data?

Did you know a program or logiciel that allows to calculate the sample size for a longitudinal study with repeated data ? And if my subjects are nested in some schools, how I calculate the sample ...
0
votes
1answer
19 views

Compute an approximate power when lambda = 10

I wanna reconfirm, is this the right way to answer this question? I'm pretty confused. And if not, how?: Answer: Thank you so much.
0
votes
0answers
9 views

Statistical Power alternatives for post-hoc analysis

I am reading about Power and how it is misleading/uninformative to calculate a post-hoc "observed power" of a study. I would still like, after reading a study, to be able to judge its ability to have ...
2
votes
1answer
28 views

Why is my sample size so big after doing pwr.t.test() in R?

For a research I want to do a t-test between two populations. Since I have a fairly small sample size I want to calculate the necessary sample-size for getting a good power (0.8 is supposed to be very ...
0
votes
0answers
12 views

Determine PPV from type I and type II error rates

Consider these two scenarios: In both of these scenarios, flipping heads 60 times (the X-axis should read # of heads, not # of coin flips) would be reported as a p of ~0.05 if we are using the fair ...
0
votes
0answers
9 views

Comparing two methods of measurement

In relation to testing a new method of measuring a variable, I am interested in knowing how accurate the new method is. With one method (M1), we measure the "ground-truth": A variable with normal ...
1
vote
1answer
44 views

Power analysis for binomial regression (success/failure)

I've collect 161 people for a study which the original power analysis was based on correlations, but now I've realised binomial regression would be better. I can find a lot of articles on comparing ...
0
votes
1answer
39 views

Required sample size to reach adequate power of a test

We got two types of treatment (type 1 and type 2). In our opinion, type 2 treatment has a higher performance and we would like to show this with research. We assume that type 1 treatment has a maximum ...
2
votes
0answers
45 views

An alternative definition of the $p$-value

As we know, $p$-values are uniformly distributed under $H_0$. This urges me to ask if this constitutes a valid (re-)definition of the $p$-value. $p$-value - A statistic with a uniform distribution ...
0
votes
0answers
6 views

How to combine Benjamini-Hochberg with statistical power?

Suppose I am investigating a question which involves many statistical T-tests. The normal Benjamini-Hochberg procedure tells me how to control the false discovery rate. However, suppose that some or ...
2
votes
2answers
118 views

Can bootstrapping improve the test power, when the sample size is small? [closed]

When the sample is small and study is under-powered, can bootstrapping be used to generate data and increase the sample size (and improve the test power)? Is there any way that bootstrapping can be ...
0
votes
0answers
20 views

Shapiro–Wilk test for non-normal distributions

The Shapiro–Wilk test makes use of the correlation between the data (a.k.a. sample quantile) $x$ and its theoretical quantile $m$. As far as I'm concerned, it's a normality test simply because the ...
0
votes
1answer
20 views

Computing the power of Welch's t-test

I constructed a Welch's t-test in order to examine the difference between the means of two samples (check whether or not one is larger than the other). $H_{0}: \mu_0 = \mu_1 ~\textrm{vs}~ H_{1}: \...
0
votes
0answers
14 views

How do I calculate the statistical reliability of a random sampling of features?

I have a dataset of 6 million features from which I've trained a binary classification model. It identified 32 features that can accurately predict 98% of samples (ie if I train a model with only ...
0
votes
0answers
11 views

How to calculate statistical power in a clustering problem?

I'm considering following question. I implement a algorithm to cluster my data and the result will assign each cluster to a small cluster or big cluster according to its size. After that, I will get ...
0
votes
0answers
31 views

Power analysis for ordinal logistic regression (multiple independent variables)

I have a question with regard to a simulation based power analysis for ordinal logistic regression suggested here: https://stats.stackexchange.com/a/22410/231675 (by Greg Snow) The suggested power ...
1
vote
2answers
78 views

proof for the hypothesis testing $\alpha-\beta$ tradeoff

All the statistic's courses state that there is a tradeoff between the significance level ($\alpha$) and the probability of the Type II error ($\beta$). Specifically, for a fixed sample size, one ...
2
votes
1answer
66 views

Significance Tests - Why Would a Low Sample Size Cause Problems?

Okay, on my AP Statistics test on the chapter on significance tests, the following free response question appeared: When the manufacturing process is working properly, NeverREady batteries have ...
0
votes
0answers
37 views

Non-Parametric Effect Sizes and Power

As my data was extremely skewed, I have ran 2 non-parametric tests: a Mann-Whitney U and a Wilcoxon Signed Rank test. I have read that using Cohen's D as an effect size for such tests is not ...
0
votes
1answer
35 views

How exactly do I calculate the power in my SEM?

I am trying to calculate the power in my SEM analysis post-hoc. How exactly should I do this? What is the power for the R-squared result of IT-T2 and IT-T3? Background info: Sample size is 255. IT ...
1
vote
0answers
41 views

UMP test for $H_0:p=0.5$ vs $H_1:p\neq0.5$?

Let $X_1,\dots, X_n$ Bernoulli trials. I know that the UMP tests for $$H_0:p=0.5 \quad\text{vs}\quad H_1:p>0.5$$ and $$H_0:p=0.5 \quad\text{vs}\quad H_1:p<0.5$$ can be obtained with the Neyman ...
0
votes
0answers
42 views

Possible to extend Gelman & Carlin's “Beyond Power Calculations” when significance not obtained?

I have recently been spending time trying to understand the content and extent/limitations of this paper: "Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors" by Gelman ...
0
votes
0answers
11 views

Formula for Power of Upper Tail Sign Test

Given Consider the upper tail $H_0: \theta \leq 0 \,\,\, \text{vs} \,\,\, H_A: \theta > 0$ sign test with the test statistic $B = \sum_{i=1}^n{\psi_i}$ where $\psi_i = \mathbb{I}(Z_i > \theta)$...
1
vote
0answers
24 views

Sample size formulae

In order to calculate sample size for hypothesis testing, we consider level of significance and power. For level of significance, two sided values ($Z_{\alpha/2}$) are taken into account, but for ...
2
votes
1answer
27 views

What causes low power on hypothesis tests?

I've been doing hypothesis tests to compare two proportions in a product. Although the sample size is supposed to be more than enough based on the population and $\alpha$ (type I error), the power of ...
4
votes
1answer
96 views

AB test sample size calculation by hand

Evan Miller has created a well-known online AB test sample size calculator. For the sake of being able to program and modify this formula, I would like to know how to calculate sample size Evan Miller-...
1
vote
0answers
26 views

Power calculations for moderator analyses in RCT

I am running a randomised controlled trial of a savings intervention in India whereby randomisation is at the individual level (not clusters). In my power calculations, I also want to factor in a ...
1
vote
1answer
31 views

Power analysis for t-test instead of ANOVA

Let's say I have an experiment with a 2x2 repeated-measures design. Let's call the four cells A1B1, A1B2, A2B1, A2B2, for the factors A and B, each with two conditions, labeled 1 and 2. I want to ...
0
votes
0answers
32 views

How do I determine the critical r value for Pearson correlation if I compute the result via bootstrapping?

The critical r value for a Pearson correlation analysis depends on sample size. For example, consider a test with the following parameters: n = 20 alpha = .05 I ...
3
votes
2answers
69 views

Can one make statistical inferences from 6 patients?

Is it conceptually correct to make statistical inferences from a very small number of patients (even when using exact methods like Fisher's test, etc)? What do I have to look out for: power or p-value?...
3
votes
2answers
830 views

Why this simple mixed model fail to converge?

I have data on which I would like to compare the mean of the value variable between day 28 and day 83. Because the experience involve pseudo-replication (...
0
votes
0answers
19 views

Sensitivity and Specificity of gaussian and negative binomial glm family

I am dealing with count data that is over-dispersed and hence I consider using the negative binomial family (with glm()) instead of the gaussian family, as the link function. To find out the ...
2
votes
0answers
25 views

How does one perform a conditional power analysis?

Suppose a test statistic $Z$ will be compared to a standard normal distribution to evaluate whether a $p$-value achieves a statistically significant result at the $\alpha$-level. Based on an ...
1
vote
1answer
34 views

Sample size needed when combining multiple t-tests

I am designing an experiment that tests a difference in means between two groups in which data points take very long to generate (1 month each). Accordingly, I'd like to run several tests of the same ...
0
votes
0answers
35 views

the power and sample size of one Poisson mean test

I was using the softeware PASS to calculate the sample size and power of a one Poisson mean problem. Solve For ................................................ Power Ha (Alternative Hypothesis).........
5
votes
1answer
40 views

Use of supportive inferential statistics (e.g. Levene's and normality tests) in underpowered samples

We know that underpowered statistics greatly increase the probability of a type II error (by definition), meaning a greater chance of failing to reject the null hypothesis despite the existence of a '...
0
votes
0answers
12 views

Type I and II errors accounted for by one equation?

On this website: https://clincalc.com/stats/samplesize.aspx both type I and II errors are accounted for by the one equation. Surely, because a type I error involves too much positive variance and a ...