# Questions tagged [power-analysis]

An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of power (e.g. $80\%$) for a given effect size. In many situation theoretical calculations are intractable so power analysis is done by simulation.

689 questions
Filter by
Sorted by
Tagged with
226k views

### Rules of thumb for minimum sample size for multiple regression

Within the context of a research proposal in the social sciences, I was asked the following question: I have always gone by 100 + m (where m is the number of predictors) when determining minimum ...
18k views

### Simulation of logistic regression power analysis - designed experiments

This question is in response to an answer given by @Greg Snow in regards to a question I asked concerning power analysis with logistic regression and SAS ...
9k views

### What percentage of a population needs a test in order to estimate prevalence of a disease? Say, COVID-19

A group of us got to discussing what percentage of a population needs to be tested for COVID-19 in order to estimate the true prevalence of the disease. It got complicated, and we ended the night (...
9k views

### Why do political polls have such large sample sizes?

When I watch the news I've noticed that the Gallup polls for things like presidential elections have [I assume random] sample sizes of well over 1,000. From what I remember from college statistics ...
26k views

### Sample size calculation for mixed models

I am wondering if there are any methods for calculating sample size in mixed models? I'm using lmer in R to fit the models (I have random slopes and intercepts).
3k views

### Is a priori power analysis essentially useless?

I attended a meeting of the Society for Personality and Social Psychology last week where I saw a talk by Uri Simonsohn with the premise that using an a priori power analysis to determine sample size ...
17k views

### Calculating statistical power

As I understand it, I need to know at least three aspects (out of four) of my proposed study in order to conduct power analysis, namely: type of test - I intend to use Pearson's r and ANCOVA/...
9k views

### Is power analysis necessary in Bayesian Statistics?

I've been researching the Bayesian take on classical statistics lately. After reading about the Bayes factor I've been left wondering if power analysis is a necessity in this view of statistics. My ...
4k views

### How does the power of a logistic regression and a t-test compare?

Is the power of a logistic regression and a t-test equivalent? If so, they should be "data density equivalent" by which I mean that the same number of underlying observations yields the same power ...
1k views

### How could one develop a stopping rule in a power analysis of two independent proportions?

I am a software developer working on A/B testing systems. I don't have a solid stats background but have been picking up knowledge over the past few months. A typical test scenario involves comparing ...
2k views

Today I noticed this question, and I thought it would be helpful if we had a thread that listed resources that people could conveniently access for power analysis / sample size calculations, perhaps ...
52k views

### Unequal sample sizes: When to call it quits

I'm peer reviewing an academic journal article and the authors wrote the following as justification for not reporting any inferential statistics (I deidentified the nature of the two groups): In ...
11k views

### Power analysis for Kruskal-Wallis or Mann-Whitney U test using R?

Is it possible to perform a power analysis for the Kruskal-Wallis and Mann-Whitney U test? If yes, are there any R packages/functions that perform it?
359 views

### Is dynamically increasing the sample size okay, if stated a priori?

I'm about to do a study about the merits of one stimulus compared to another with a within-subject design. I have a permutation scheme that is designed to reduce order effects of some parts of the ...
5k views

### Power of FDR vs. FWER approaches in multiple comparisons [duplicate]

Regarding multiple comparisons, could someone please explain me why the power of false discovery rate (FDR) is greater than the power of family-wise error rate (FWER)?
4k views

### Sample size needed to estimate probability of “success” in Bernoulli trial

Suppose a game offers an event which upon completion, either gives a reward, or gives nothing. The exact mechanism for determining whether the reward is given is unknown, but I assume a random number ...
4k views

### How to simulate a custom power analysis of an lm model (using R)

Following to the recent questions we had here. I was hopping to know if anyone had come across or can share R code for performing a custom power analysis based on simulation for a linear model? ...
9k views

### Power analysis for ordinal logistic regression

I am looking for a program (in R or SAS or standalone, if free or low cost) that will do power analysis for ordinal logistic regression.
34k views

### How do I find the probability of a type II error?

I know that a Type II error is where H1 is true, but H0 is not rejected. Question How do I calculate the probability of a Type II error involving a normal distribution, where the standard deviation ...
2k views

### Minimum sample size per cluster in a random effect model

Is there a rational for the number of observations per cluster in a random effect model? I have a sample size of 1,500 with 700 clusters modeled as exchangeable random effect. I have the option to ...
7k views

### What does it mean for a study to be over-powered?

What does it mean for a study to be over-powered? My impression is that it means that your sample sizes are so large that you have the power to detect minuscule effect sizes. These effect sizes are ...
4k views

### Power analysis from Bayesian point of view [duplicate]

I was reading an article on a power analysis and sample size calculations for a class of models that known as joint longitudinal-survival models. The paper derives a closed form formula for power and ...
603 views

### When (if ever) is it a good idea to do a post hoc power analysis?

My understanding is that a power analysis is post hoc if and only if it uses the observed effect size as the target population effect size.
13k views

### Power analysis for binomial data when the null hypothesis is that $p = 0$

I'd like to do a power analysis for a single sample from binomial data, with $H_0: p = 0$, vs. $H_1: p = 0.001$, where $p$ is the proportion of successes in the population. If $0 < p <1$, I ...
2k views

### What is the power of the regression F test?

The classical F-test for subsets of variables in multilinear regression has the form $$F = \frac{(\mbox{SSE}(R) - \mbox{SSE}(B))/(df_R - df_B)}{\mbox{SSE}(B)/df_B},$$ where $\mbox{SSE}(R)$ is the ...
8k views

### Noncentrality Parameter - what is it, what does it do, what would be a suggested value?

I have been trying to brush up on my stats knowledge, especially in relation to Sample size determination and Statistical Power Analysis. But it seems that the more I read the more I need to read. ...
1k views

### What are the ''desirable'' statistical properties of the likelihood ratio test?

I am reading an article whose method is fully based on the likelihood ratio test. The author says that the LR test against one sided alternatives is UMP. He proceeds by claiming that "...even when ...
2k views

### Power analysis for survival analysis

If I hypothesize that a gene signature will identify subjects at a lower risk of recurrence, that is decrease by 0.5 (hazard ratio of 0.5) the event rate in 20% of the population and I intend to use ...
2k views

### How to use G Power 3 to calculate statistical power in mixed design ANOVA with unequal group sample sizes [closed]

In G power 3, ANOVA repeated measures within-between interaction: Only the total sample size is reported assuming equal sample size for the two groups. My questions are: How would it work if the ...
4k views

### Can you calculate the power of a Kolmogorov-Smirnov test in R?

Is it possible to do a power analysis for a 2-sided Kolmogorov Smirnov test in R? I am testing whether two empirical distributions differ using ks.test(), and am looking to add a power analysis. I ...
8k views

### Can I do a t-test Power Analysis for Unequal Size Groups which Produces 2 Different Minimum n's?

It's usually straightforward to do a Power Analysis to compute the minimum sample size, especially in R which is my preferred ...
10k views

### Power for two sample t test

I am trying to understand power calculation for the case of the two independent sample t-test (not assuming equal variances so I used Satterthwaite). Here is a diagram that I found to help understand ...
2k views

### Determining sample size with a proportion and binomial distribution

I am trying to learn some statistics using the book, Biometry by Sokal and Rohlf (3e). This is an exercise in the 5th chapter which covers probability, the binomial distribution, and Poisson ...
3k views

### Two-sample comparison of proportions, sample size estimation: R vs Stata

Two-sample comparison of proportions, sample size estimation: R vs Stata I got different results for sample sizes, as follows: In R ...
10k views

### Required number of simulations for Monte Carlo analysis

My question is about the required number of simulations for Monte Carlo analysis method. As far as I see the required number of simulations for any allowed percentage error $E$ (e.g., 5) is  n = \...
2k views

### Assessing the power of a normality test (in R)

I want to assess the accuracy of normality tests over different sample sizes in R (I realize that normality tests may be misleading). For example, to look at the Shapiro-Wilk test, I'm conducting the ...
2k views

### In a meta-analysis, how should one handle non-significant studies containing no raw data?

Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
18k views

### SD larger than mean, non-negative scale

I was given an article reporting a study very similar to one my lab wishes to run. But, I noticed that for the variable of interest, Duration, the SDs are larger than the mean...since this is ...
99 views

### Does imputation introduce unacceptable bias?

I have recently come to know about imputation techniques, which, in short, "guess" realistic values with which to replace missing values in a dataset. My big issue with this is that we are ...
638 views

### Power in proteomics?

Grants often require power analysis to support a proposed sample size. In proteomics (and most -omics), there are 100's to 1000's of features/variables measured across 10's of samples (maybe 100s, but ...
578 views

### Consequences of exceeding sample size after determination of sample in power analysis

Challenge In our office, I have been involved in a discussion about sample size and its influence on effect size - can you help me out and explain further? Base When conducting a power analysis, ...
2k views

### Sample size calculation for paired ordinal data

What formula should be used for sample size calculations for paired ordinal data? Imagine we have an intervention and we want to measure the effect based on before and after measurements of an ordered ...
220 views

### How can one perform a two-group binomial power analysis without using normal approximations?

I would like to do power analyses for hypothesis tests of (non-)equality of proportions in which the proportions are very small. I would like to do so without using normal (or Poisson) approximations ...
887 views

### Why is power of a hypothesis test a concern when we can bootstrap any representative sample to make n approach infinity?

Why do we care about the power of a hypothesis test if we no longer live in an age where computers are slow and it's too costly to bootstrap/do a permutation test on anything which is also non-...
19k views

### Power of the t-test under unequal sample sizes

I heard that the power of a t-test used with unequal samples is limited by the size of the smaller sample. Can I take this to mean that the power of a t-test with unequal sample sizes is equal to the ...
2k views

### What is the impact of doubling a sample size on a p-value

Assuming that their is an underlying relationship between two variables in an OLS regression [null hypothesis testing], then what is the impact on the p-value of doubling the sample size? (assuming ...
5k views

### Differences and relation between retrospective power analysis and a posteriori power analysis?

From a note A Priori Power Analysis. This is an important part of planning research. You determine how many cases you will need to have a good chance of detecting an effect of a specified size ...
134 views

### Why don't people trade significance level for power?

As a convention, we have a lot of studies whose significance level is $0.05$ and a power of $0.8$. However, it is extremely rare to find a study whose $\alpha = 0.2$ with a power of $0.95$. From my ...
2k views

### Desired effect size vs. expected effect size

When estimating the effect size for determining optimal sample size for a statistical test, the textbook approach seems to do this from pilot studies or the like. Is it also legal to define effect ...