# Questions tagged [power-analysis]

An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of power (e.g. $80\%$) for a given effect size. In many situation theoretical calculations are intractable so power analysis is done by simulation.

556 questions
0answers
20 views

### Independent Sample Power Analysis With Unequal Means

I am running a test with two groups that have equal sample sizes (33 per group) but very unequal means - control group: 28,000 vs test group: 80,000. I am trying to determine the required sample ...
1answer
18 views

### Resource for an explanation of Sample size calculation when $\alpha$, Power and Effect Size for t-Test are given

For a simple z-Test or t-Test (one-sample or two-independent-sample with $n_1 = n_2$ and equal variance) I want to calculate the required sample size $n$ when the significance level $\alpha$, power ...
1answer
16 views

### Power analysis in a clinical trial

In a randomized clinical trial, is power analysis based on observed means of the treatment groups or true mean of the treatments groups?
0answers
20 views

### What is a computationally tractable way to infer statistical power in the context of OLS with robust clustered standard errors?

I have clustered data and am trying to scale up OLS with robust clustered standard errors (implemented in Python Statsmodel) as the default analytical framework going forward. It is important that I ...
0answers
4 views

### When calculating my sample size, or my power, using a power.anova.test in r, why must I square my within group variances?

I am trying to run an experiment/observational study where I am measuring the diameter of glass bottles and I am trying to calculate my sample size for many different groups. I have found that my ...
0answers
20 views

### Power analysis for prospective cohort study

I am writing a grant proposal for a project which primarily aims to test the association between vaginal microbiota composition and miscarriage in a prospective, population-based cohort of pregnant ...
1answer
22 views

### What is necessary sample size for B in A/B test with unequal sample sizes?

I have a situation in which I want to measure if the error rate in system B (test) is worse than the error rate in system A (control). System A has a sample size of 100 and that cannot be modified. So ...
1answer
14 views

### Logistic regression: Sample size for emmeans comparisons

I conducted a logistic regression with a 3-levels categorical predictor in SPSS using the GENLIN procedure. As I am interested in the pairwise comparisons between all three predictor levels, I used ...
0answers
10 views

### How does variation in the effect size affects power?

Effect size The effect size affect the power of a statistical test. We typically summarizes the magnitude of the effect of a variable as a single number (which we call the effect size). To my ...
1answer
34 views

### Standard Error over the mean: a valid 'modified coefficient of variation'?

At my work, we have used a 'modified Coefficient of Variation', defined as the Standard Error over the mean, and I wanted to ask if there is any statistical justification for this. We are interested ...
1answer
24 views

### Power calculations

I am using a toy example to explain power calculations for a t-test Let's say we have two populations where we measure a certain parameter. Population A has a mean of 0 and population B has a mean of ...
0answers
33 views

### What's the right way to make a null distribution?

I want to run an a/b test. I'd like to check if my results in the experimental variant are extreme, under the null hypothesis. I haven't created my sample, yet, and I'd like to do a power analysis to ...
0answers
9 views

### Statistical Power alternatives for post-hoc analysis

I am reading about Power and how it is misleading/uninformative to calculate a post-hoc "observed power" of a study. I would still like, after reading a study, to be able to judge its ability to have ...
0answers
13 views

### Statistical power and number of independent variables

I would like to understand properly the idea that increasing the number of independent variables in a linear regression decreases the statistical power of the estimated parameters. I have to include ...
1answer
43 views

### Power analysis for binomial regression (success/failure)

I've collect 161 people for a study which the original power analysis was based on correlations, but now I've realised binomial regression would be better. I can find a lot of articles on comparing ...
0answers
46 views

### Using power.t.test() to calculate n for sampling population mean

I'd like to know the required sample size needed to sample a population within 14.7 units of its mean, where the sd = 46, alpha = 0.01, and power = 0.99. I try R's ...
0answers
6 views

### Signal detection theory: one-interval design (= yes-no task) and 2AFC design, test power and sample size

my research group and I read some more about signal detection theory and we were not able to find answers concerning the following questions. Therefore, we would appreciate your help very much! 1) We ...
0answers
30 views

### Experimental Design Fixed Effect Minimal Detectable Difference

How does one determine the minimal detectable difference for a fixed factor/interaction in a general balanced experimental design? For example, I would like to be able to recreate Appendix 12 of ...
2answers
114 views

### Can bootstrapping improve the test power, when the sample size is small? [closed]

When the sample is small and study is under-powered, can bootstrapping be used to generate data and increase the sample size (and improve the test power)? Is there any way that bootstrapping can be ...
1answer
17 views

### Power analysis: number of participants or number of data points?

This almost answered my question below, but not quite. I want to calculate the sample size required in order to reach a certain level of a priori statistical power in my experiment. My question is ...
0answers
8 views

### Estimate sample size in respect to percent of difference between two means

I am trying to find a way to estimate (even in a crude manner) what sample size I need to detect a mean difference of say 5% between two independent groups. Problem is, I do not know sigma nor there ...
0answers
14 views

### How do I calculate the statistical reliability of a random sampling of features?

I have a dataset of 6 million features from which I've trained a binary classification model. It identified 32 features that can accurately predict 98% of samples (ie if I train a model with only ...
0answers
37 views

### Calculating power for balanced incomplete block design

My experiment has a factorial 2x2 design: 2 factors are manipulated at levels 'high' and 'low'. We thus get 4 conditions: Condition 1: Factor A high, Factor B high Condition 2: Factor A high, Factor ...
0answers
11 views

### How to calculate statistical power in a clustering problem?

I'm considering following question. I implement a algorithm to cluster my data and the result will assign each cluster to a small cluster or big cluster according to its size. After that, I will get ...
1answer
51 views

### (Sample Size calculation) G*Power analysis: ANCOVA

I wrote a similar post just before, but I realized that I put wrong information there, hence I deleted it, and have re-written here I am trying to conduct a power analysis on a hierarchical ...
1answer
38 views

### Power analysis for same model as “ANOVA” vs as “multiple regression” yields different results

I have seen posts that said ANOVA and multiple regression are theoretically the same. But if this is really the case, does anyone know why the G*Power (Linear multiple regression vs ANOVA) gives ...
0answers
20 views

### Minimum sample size calculation a posteriori

I am running a number of experiments where I obtain a large number of observations with one variable (yielding two sets of values; many more than would be needed for a hypothesis test). I want to test ...
0answers
16 views

### Sample Size: G*Power for hierarchical multiple regression with a three-way interaction?

I have been struggling quite a lot with finding a way to conduct a power analysis for a hierarchical regression with a three-way interaction. I will put 2 control variables in Step 1, 3 IVs in Step 2 ...
1answer
28 views

### Sample size needed to prove equal proportions

We are trying to prove that three groups have the same proportion of some effect. We are struggling with formulating the null hypothesis, and whether or not the sample size is big enough. The ...
1answer
47 views

### Power of paired t-test: R vs. SAS

If we are trying to find the power of a paired t-test, SAS offers this template (among others) in PROC POWER: ...
0answers
95 views

### Sample size calculation in repeated measures design for linear mixed models

Does anyone know a method how to calculate the required sample size in a repeated measures design that will be analysed using linear mixed modelling? The design looks as follows:I have two groups (...
0answers
25 views

### Is 1/5 of studies that use 80% power calculations wrong?

I'm just learning about power calculations. As I understood, the most commonly used power is 80%, meaning that if there is an effect, the null hypothesis is rejected in 80% of the cases. However, I'm ...
1answer
38 views

### Power analysis for multilevel model with two fixed and two random (crossed) effects: how to define sample size with simr package [closed]

I want to define the sample size for a repeated measurement experiment. Every subject will indicate her/his behavior on a 1-7 scale for several stimuli (=behavior). I expect that personality and ...
0answers
15 views

### Power Analysis for Case Control Regression?

If we conduct a case control study and plan to analyze the results with multiple linear regression, which power analysis should be conducted to select sample size? There are separate calculators for ...
1answer
45 views

### Determining a Statistically Significant Result

I feel like this is an easy question if I just knew the right words to Google, but apparently I don't, so I'm sorry if it's been asked before. I'm simply looking to determine how long I need to ...
1answer
27 views

### What causes low power on hypothesis tests?

I've been doing hypothesis tests to compare two proportions in a product. Although the sample size is supposed to be more than enough based on the population and $\alpha$ (type I error), the power of ...
1answer
30 views

### Power analysis for t-test instead of ANOVA

Let's say I have an experiment with a 2x2 repeated-measures design. Let's call the four cells A1B1, A1B2, A2B1, A2B2, for the factors A and B, each with two conditions, labeled 1 and 2. I want to ...
1answer
23 views

### Sample size needed?

I need some help calculating the minimum sample size needed for my upcoming trial. I’m doing higher degree research on eHealth, so I have little knowledge about statistics and sample size calculation. ...
0answers
25 views

### Power Analysis and Non-Central t Distribution

I'm a bit confused about power analysis and the non-central t-distribution. If I am in a situation where $H_0$ is $\mu = 0$, and $H_1$ is $\mu > 0$, I calculate for what value $\mu$ would have a ...
0answers
18 views

### Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
0answers
17 views

### Power calculation for TSSEM or Meta-analytic SEM

I am wondering about power calculations for Cheung's metaSEM/TTSEM. I assume Stage 2 SEM is where having too few studies/small sample sizes becomes problematic. Is there a way to run an a priori ...
1answer
16 views

### Help with calculating number of animals for a fetal surgery experiment

Rather odd question. I'm a surgical resident who is currently a research fellow in a lab studying fetal surgery. Our plan is to test a glue which we believe will help seal a uterus after we open it to ...
2answers
41 views

### Statistic metric for groups with different observations

I have this problem and more I read online, more confused I'm getting. So I have 4 (or more) groups with different users (for simplicity I only used small numbers, but I may have 50, 100, 150 and 200 ...
2answers
34 views

### Is it the case that β+power=1?

In psychology papers that do prospective power analyses (example), one often notices the convention of assuming β values (false negative rates) of 0.20, and power levels of 0.80. In other words (if I ...
0answers
42 views

### Power of Chi-squared test in R

I am trying to calculate the required sample size for my study. I found the pwr-package to be useful but cannot reproduce the results. What I did: ...
1answer
53 views

### Is there a relationship between number of covariates and sample size in logistic regression? [duplicate]

Is there some definite relationship between number of covariates and the sample size in logistic regression? (e.g. larger the number of covariates, larger the sample size needed, etc.) Thank you,
2answers
252 views

### Minimum sample size per cluster in a random effect model

Is there a rational for the number of observations per cluster in a random effect model? I have a sample size of 1,500 with 700 clusters modeled as exchangeable random effect. I have the option to ...
0answers
24 views

### Power Analysis with Noncentral F distribution

I'm looking to replicate a power analysis for binary data using an MRMC analysis (details found here). The data looks like this, where Score is a binary variable. ...
1answer
133 views

### How to conduct power analysis for unbalanced one-way ANOVA in R?

I am looking for a way to conduct power analysis for unbalanced one-way ANOVA in R. I found the pwd package; however, it only allows me to calculate power for ...
0answers
20 views

### Sample size calculation for parametric test, with large effect expected, small numbers. Should I use parametric or non parametric tests?

I am trying to figure out what would be the best way to analyze data from a randomized double blind trial we conducted. We sought to find if two dosages of a drug were effective for a severe symptom ...