Questions tagged [power-analysis]

An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of power (e.g. $80\%$) for a given effect size. In many situation theoretical calculations are intractable so power analysis is done by simulation.

Filter by
Sorted by
Tagged with
75
votes
7answers
226k views

Rules of thumb for minimum sample size for multiple regression

Within the context of a research proposal in the social sciences, I was asked the following question: I have always gone by 100 + m (where m is the number of predictors) when determining minimum ...
39
votes
2answers
18k views

Simulation of logistic regression power analysis - designed experiments

This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
37
votes
3answers
9k views

What percentage of a population needs a test in order to estimate prevalence of a disease? Say, COVID-19

A group of us got to discussing what percentage of a population needs to be tested for COVID-19 in order to estimate the true prevalence of the disease. It got complicated, and we ended the night (...
33
votes
5answers
9k views

Why do political polls have such large sample sizes?

When I watch the news I've noticed that the Gallup polls for things like presidential elections have [I assume random] sample sizes of well over 1,000. From what I remember from college statistics ...
24
votes
3answers
26k views

Sample size calculation for mixed models

I am wondering if there are any methods for calculating sample size in mixed models? I'm using lmer in R to fit the models (I have random slopes and intercepts).
23
votes
1answer
3k views

Is a priori power analysis essentially useless?

I attended a meeting of the Society for Personality and Social Psychology last week where I saw a talk by Uri Simonsohn with the premise that using an a priori power analysis to determine sample size ...
22
votes
1answer
17k views

Calculating statistical power

As I understand it, I need to know at least three aspects (out of four) of my proposed study in order to conduct power analysis, namely: type of test - I intend to use Pearson's r and ANCOVA/...
21
votes
4answers
9k views

Is power analysis necessary in Bayesian Statistics?

I've been researching the Bayesian take on classical statistics lately. After reading about the Bayes factor I've been left wondering if power analysis is a necessity in this view of statistics. My ...
19
votes
2answers
4k views

How does the power of a logistic regression and a t-test compare?

Is the power of a logistic regression and a t-test equivalent? If so, they should be "data density equivalent" by which I mean that the same number of underlying observations yields the same power ...
19
votes
4answers
1k views

How could one develop a stopping rule in a power analysis of two independent proportions?

I am a software developer working on A/B testing systems. I don't have a solid stats background but have been picking up knowledge over the past few months. A typical test scenario involves comparing ...
18
votes
1answer
2k views

Free internet or downloadable resources for sample size calculations

Today I noticed this question, and I thought it would be helpful if we had a thread that listed resources that people could conveniently access for power analysis / sample size calculations, perhaps ...
14
votes
2answers
52k views

Unequal sample sizes: When to call it quits

I'm peer reviewing an academic journal article and the authors wrote the following as justification for not reporting any inferential statistics (I deidentified the nature of the two groups): In ...
14
votes
2answers
11k views

Power analysis for Kruskal-Wallis or Mann-Whitney U test using R?

Is it possible to perform a power analysis for the Kruskal-Wallis and Mann-Whitney U test? If yes, are there any R packages/functions that perform it?
13
votes
3answers
359 views

Is dynamically increasing the sample size okay, if stated a priori?

I'm about to do a study about the merits of one stimulus compared to another with a within-subject design. I have a permutation scheme that is designed to reduce order effects of some parts of the ...
13
votes
1answer
5k views

Power of FDR vs. FWER approaches in multiple comparisons [duplicate]

Regarding multiple comparisons, could someone please explain me why the power of false discovery rate (FDR) is greater than the power of family-wise error rate (FWER)?
13
votes
3answers
4k views

Sample size needed to estimate probability of “success” in Bernoulli trial

Suppose a game offers an event which upon completion, either gives a reward, or gives nothing. The exact mechanism for determining whether the reward is given is unknown, but I assume a random number ...
13
votes
3answers
4k views

How to simulate a custom power analysis of an lm model (using R)

Following to the recent questions we had here. I was hopping to know if anyone had come across or can share R code for performing a custom power analysis based on simulation for a linear model? ...
12
votes
4answers
9k views

Power analysis for ordinal logistic regression

I am looking for a program (in R or SAS or standalone, if free or low cost) that will do power analysis for ordinal logistic regression.
12
votes
2answers
34k views

How do I find the probability of a type II error?

I know that a Type II error is where H1 is true, but H0 is not rejected. Question How do I calculate the probability of a Type II error involving a normal distribution, where the standard deviation ...
12
votes
2answers
2k views

Minimum sample size per cluster in a random effect model

Is there a rational for the number of observations per cluster in a random effect model? I have a sample size of 1,500 with 700 clusters modeled as exchangeable random effect. I have the option to ...
12
votes
4answers
7k views

What does it mean for a study to be over-powered?

What does it mean for a study to be over-powered? My impression is that it means that your sample sizes are so large that you have the power to detect minuscule effect sizes. These effect sizes are ...
12
votes
1answer
4k views

Power analysis from Bayesian point of view [duplicate]

I was reading an article on a power analysis and sample size calculations for a class of models that known as joint longitudinal-survival models. The paper derives a closed form formula for power and ...
12
votes
2answers
603 views

When (if ever) is it a good idea to do a post hoc power analysis?

My understanding is that a power analysis is post hoc if and only if it uses the observed effect size as the target population effect size.
11
votes
3answers
13k views

Power analysis for binomial data when the null hypothesis is that $p = 0$

I'd like to do a power analysis for a single sample from binomial data, with $H_0: p = 0$, vs. $H_1: p = 0.001$, where $p$ is the proportion of successes in the population. If $0 < p <1$, I ...
11
votes
2answers
2k views

What is the power of the regression F test?

The classical F-test for subsets of variables in multilinear regression has the form $$ F = \frac{(\mbox{SSE}(R) - \mbox{SSE}(B))/(df_R - df_B)}{\mbox{SSE}(B)/df_B}, $$ where $\mbox{SSE}(R)$ is the ...
11
votes
1answer
8k views

Noncentrality Parameter - what is it, what does it do, what would be a suggested value?

I have been trying to brush up on my stats knowledge, especially in relation to Sample size determination and Statistical Power Analysis. But it seems that the more I read the more I need to read. ...
11
votes
1answer
1k views

What are the ''desirable'' statistical properties of the likelihood ratio test?

I am reading an article whose method is fully based on the likelihood ratio test. The author says that the LR test against one sided alternatives is UMP. He proceeds by claiming that "...even when ...
11
votes
1answer
2k views

Power analysis for survival analysis

If I hypothesize that a gene signature will identify subjects at a lower risk of recurrence, that is decrease by 0.5 (hazard ratio of 0.5) the event rate in 20% of the population and I intend to use ...
11
votes
0answers
2k views

How to use G Power 3 to calculate statistical power in mixed design ANOVA with unequal group sample sizes [closed]

In G power 3, ANOVA repeated measures within-between interaction: Only the total sample size is reported assuming equal sample size for the two groups. My questions are: How would it work if the ...
10
votes
2answers
4k views

Can you calculate the power of a Kolmogorov-Smirnov test in R?

Is it possible to do a power analysis for a 2-sided Kolmogorov Smirnov test in R? I am testing whether two empirical distributions differ using ks.test(), and am looking to add a power analysis. I ...
10
votes
2answers
8k views

Can I do a t-test Power Analysis for Unequal Size Groups which Produces 2 Different Minimum n's?

It's usually straightforward to do a Power Analysis to compute the minimum sample size, especially in R which is my preferred ...
10
votes
2answers
10k views

Power for two sample t test

I am trying to understand power calculation for the case of the two independent sample t-test (not assuming equal variances so I used Satterthwaite). Here is a diagram that I found to help understand ...
10
votes
1answer
2k views

Determining sample size with a proportion and binomial distribution

I am trying to learn some statistics using the book, Biometry by Sokal and Rohlf (3e). This is an exercise in the 5th chapter which covers probability, the binomial distribution, and Poisson ...
10
votes
1answer
3k views

Two-sample comparison of proportions, sample size estimation: R vs Stata

Two-sample comparison of proportions, sample size estimation: R vs Stata I got different results for sample sizes, as follows: In R ...
10
votes
1answer
10k views

Required number of simulations for Monte Carlo analysis

My question is about the required number of simulations for Monte Carlo analysis method. As far as I see the required number of simulations for any allowed percentage error $E$ (e.g., 5) is $$ n = \...
9
votes
3answers
2k views

Assessing the power of a normality test (in R)

I want to assess the accuracy of normality tests over different sample sizes in R (I realize that normality tests may be misleading). For example, to look at the Shapiro-Wilk test, I'm conducting the ...
9
votes
2answers
2k views

In a meta-analysis, how should one handle non-significant studies containing no raw data?

Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
9
votes
2answers
18k views

SD larger than mean, non-negative scale

I was given an article reporting a study very similar to one my lab wishes to run. But, I noticed that for the variable of interest, Duration, the SDs are larger than the mean...since this is ...
9
votes
1answer
99 views

Does imputation introduce unacceptable bias?

I have recently come to know about imputation techniques, which, in short, "guess" realistic values with which to replace missing values in a dataset. My big issue with this is that we are ...
9
votes
1answer
638 views

Power in proteomics?

Grants often require power analysis to support a proposed sample size. In proteomics (and most -omics), there are 100's to 1000's of features/variables measured across 10's of samples (maybe 100s, but ...
8
votes
1answer
578 views

Consequences of exceeding sample size after determination of sample in power analysis

Challenge In our office, I have been involved in a discussion about sample size and its influence on effect size - can you help me out and explain further? Base When conducting a power analysis, ...
8
votes
2answers
2k views

Sample size calculation for paired ordinal data

What formula should be used for sample size calculations for paired ordinal data? Imagine we have an intervention and we want to measure the effect based on before and after measurements of an ordered ...
8
votes
1answer
220 views

How can one perform a two-group binomial power analysis without using normal approximations?

I would like to do power analyses for hypothesis tests of (non-)equality of proportions in which the proportions are very small. I would like to do so without using normal (or Poisson) approximations ...
7
votes
1answer
887 views

Why is power of a hypothesis test a concern when we can bootstrap any representative sample to make n approach infinity?

Why do we care about the power of a hypothesis test if we no longer live in an age where computers are slow and it's too costly to bootstrap/do a permutation test on anything which is also non-...
7
votes
2answers
19k views

Power of the t-test under unequal sample sizes

I heard that the power of a t-test used with unequal samples is limited by the size of the smaller sample. Can I take this to mean that the power of a t-test with unequal sample sizes is equal to the ...
7
votes
3answers
2k views

What is the impact of doubling a sample size on a p-value

Assuming that their is an underlying relationship between two variables in an OLS regression [null hypothesis testing], then what is the impact on the p-value of doubling the sample size? (assuming ...
7
votes
2answers
5k views

Differences and relation between retrospective power analysis and a posteriori power analysis?

From a note A Priori Power Analysis. This is an important part of planning research. You determine how many cases you will need to have a good chance of detecting an effect of a specified size ...
7
votes
3answers
134 views

Why don't people trade significance level for power?

As a convention, we have a lot of studies whose significance level is $0.05$ and a power of $0.8$. However, it is extremely rare to find a study whose $\alpha = 0.2$ with a power of $0.95$. From my ...
7
votes
2answers
2k views

Desired effect size vs. expected effect size

When estimating the effect size for determining optimal sample size for a statistical test, the textbook approach seems to do this from pilot studies or the like. Is it also legal to define effect ...
7
votes
1answer
904 views

Calculation of power of survival study

For an upcoming study of about 200 (rare) cancer cases, we would like to determine the power of detection for a hypothetical marker, present in 10% of cases, i.e. 20 cases with a hazard ratio of 3.0. ...

1
2 3 4 5
14