Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [power-analysis]

An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of power (e.g. $80\%$) for a given effect size. In many situation theoretical calculations are intractable so power analysis is done by simulation.

0
votes
0answers
20 views

Independent Sample Power Analysis With Unequal Means

I am running a test with two groups that have equal sample sizes (33 per group) but very unequal means - control group: 28,000 vs test group: 80,000. I am trying to determine the required sample ...
0
votes
1answer
18 views

Resource for an explanation of Sample size calculation when $\alpha$, Power and Effect Size for t-Test are given

For a simple z-Test or t-Test (one-sample or two-independent-sample with $n_1 = n_2$ and equal variance) I want to calculate the required sample size $n$ when the significance level $\alpha$, power ...
0
votes
1answer
16 views

Power analysis in a clinical trial

In a randomized clinical trial, is power analysis based on observed means of the treatment groups or true mean of the treatments groups?
0
votes
0answers
20 views

What is a computationally tractable way to infer statistical power in the context of OLS with robust clustered standard errors?

I have clustered data and am trying to scale up OLS with robust clustered standard errors (implemented in Python Statsmodel) as the default analytical framework going forward. It is important that I ...
0
votes
0answers
4 views

When calculating my sample size, or my power, using a power.anova.test in r, why must I square my within group variances?

I am trying to run an experiment/observational study where I am measuring the diameter of glass bottles and I am trying to calculate my sample size for many different groups. I have found that my ...
0
votes
0answers
20 views

Power analysis for prospective cohort study

I am writing a grant proposal for a project which primarily aims to test the association between vaginal microbiota composition and miscarriage in a prospective, population-based cohort of pregnant ...
0
votes
1answer
22 views

What is necessary sample size for B in A/B test with unequal sample sizes?

I have a situation in which I want to measure if the error rate in system B (test) is worse than the error rate in system A (control). System A has a sample size of 100 and that cannot be modified. So ...
0
votes
1answer
14 views

Logistic regression: Sample size for emmeans comparisons

I conducted a logistic regression with a 3-levels categorical predictor in SPSS using the GENLIN procedure. As I am interested in the pairwise comparisons between all three predictor levels, I used ...
0
votes
0answers
10 views

How does variation in the effect size affects power?

Effect size The effect size affect the power of a statistical test. We typically summarizes the magnitude of the effect of a variable as a single number (which we call the effect size). To my ...
0
votes
1answer
34 views

Standard Error over the mean: a valid 'modified coefficient of variation'?

At my work, we have used a 'modified Coefficient of Variation', defined as the Standard Error over the mean, and I wanted to ask if there is any statistical justification for this. We are interested ...
3
votes
1answer
24 views

Power calculations

I am using a toy example to explain power calculations for a t-test Let's say we have two populations where we measure a certain parameter. Population A has a mean of 0 and population B has a mean of ...
0
votes
0answers
33 views

What's the right way to make a null distribution?

I want to run an a/b test. I'd like to check if my results in the experimental variant are extreme, under the null hypothesis. I haven't created my sample, yet, and I'd like to do a power analysis to ...
0
votes
0answers
9 views

Statistical Power alternatives for post-hoc analysis

I am reading about Power and how it is misleading/uninformative to calculate a post-hoc "observed power" of a study. I would still like, after reading a study, to be able to judge its ability to have ...
0
votes
0answers
13 views

Statistical power and number of independent variables

I would like to understand properly the idea that increasing the number of independent variables in a linear regression decreases the statistical power of the estimated parameters. I have to include ...
1
vote
1answer
43 views

Power analysis for binomial regression (success/failure)

I've collect 161 people for a study which the original power analysis was based on correlations, but now I've realised binomial regression would be better. I can find a lot of articles on comparing ...
0
votes
0answers
46 views

Using power.t.test() to calculate n for sampling population mean

I'd like to know the required sample size needed to sample a population within 14.7 units of its mean, where the sd = 46, alpha = 0.01, and power = 0.99. I try R's ...
0
votes
0answers
6 views

Signal detection theory: one-interval design (= yes-no task) and 2AFC design, test power and sample size

my research group and I read some more about signal detection theory and we were not able to find answers concerning the following questions. Therefore, we would appreciate your help very much! 1) We ...
0
votes
0answers
30 views

Experimental Design Fixed Effect Minimal Detectable Difference

How does one determine the minimal detectable difference for a fixed factor/interaction in a general balanced experimental design? For example, I would like to be able to recreate Appendix 12 of ...
2
votes
2answers
114 views

Can bootstrapping improve the test power, when the sample size is small? [closed]

When the sample is small and study is under-powered, can bootstrapping be used to generate data and increase the sample size (and improve the test power)? Is there any way that bootstrapping can be ...
1
vote
1answer
17 views

Power analysis: number of participants or number of data points?

This almost answered my question below, but not quite. I want to calculate the sample size required in order to reach a certain level of a priori statistical power in my experiment. My question is ...
0
votes
0answers
8 views

Estimate sample size in respect to percent of difference between two means

I am trying to find a way to estimate (even in a crude manner) what sample size I need to detect a mean difference of say 5% between two independent groups. Problem is, I do not know sigma nor there ...
0
votes
0answers
14 views

How do I calculate the statistical reliability of a random sampling of features?

I have a dataset of 6 million features from which I've trained a binary classification model. It identified 32 features that can accurately predict 98% of samples (ie if I train a model with only ...
0
votes
0answers
37 views

Calculating power for balanced incomplete block design

My experiment has a factorial 2x2 design: 2 factors are manipulated at levels 'high' and 'low'. We thus get 4 conditions: Condition 1: Factor A high, Factor B high Condition 2: Factor A high, Factor ...
0
votes
0answers
11 views

How to calculate statistical power in a clustering problem?

I'm considering following question. I implement a algorithm to cluster my data and the result will assign each cluster to a small cluster or big cluster according to its size. After that, I will get ...
0
votes
1answer
51 views

(Sample Size calculation) G*Power analysis: ANCOVA

I wrote a similar post just before, but I realized that I put wrong information there, hence I deleted it, and have re-written here I am trying to conduct a power analysis on a hierarchical ...
1
vote
1answer
38 views

Power analysis for same model as “ANOVA” vs as “multiple regression” yields different results

I have seen posts that said ANOVA and multiple regression are theoretically the same. But if this is really the case, does anyone know why the G*Power (Linear multiple regression vs ANOVA) gives ...
0
votes
0answers
20 views

Minimum sample size calculation a posteriori

I am running a number of experiments where I obtain a large number of observations with one variable (yielding two sets of values; many more than would be needed for a hypothesis test). I want to test ...
0
votes
0answers
16 views

Sample Size: G*Power for hierarchical multiple regression with a three-way interaction?

I have been struggling quite a lot with finding a way to conduct a power analysis for a hierarchical regression with a three-way interaction. I will put 2 control variables in Step 1, 3 IVs in Step 2 ...
1
vote
1answer
28 views

Sample size needed to prove equal proportions

We are trying to prove that three groups have the same proportion of some effect. We are struggling with formulating the null hypothesis, and whether or not the sample size is big enough. The ...
1
vote
1answer
47 views

Power of paired t-test: R vs. SAS

If we are trying to find the power of a paired t-test, SAS offers this template (among others) in PROC POWER: ...
0
votes
0answers
95 views

Sample size calculation in repeated measures design for linear mixed models

Does anyone know a method how to calculate the required sample size in a repeated measures design that will be analysed using linear mixed modelling? The design looks as follows:I have two groups (...
1
vote
0answers
25 views

Is 1/5 of studies that use 80% power calculations wrong?

I'm just learning about power calculations. As I understood, the most commonly used power is 80%, meaning that if there is an effect, the null hypothesis is rejected in 80% of the cases. However, I'm ...
1
vote
1answer
38 views

Power analysis for multilevel model with two fixed and two random (crossed) effects: how to define sample size with simr package [closed]

I want to define the sample size for a repeated measurement experiment. Every subject will indicate her/his behavior on a 1-7 scale for several stimuli (=behavior). I expect that personality and ...
0
votes
0answers
15 views

Power Analysis for Case Control Regression?

If we conduct a case control study and plan to analyze the results with multiple linear regression, which power analysis should be conducted to select sample size? There are separate calculators for ...
3
votes
1answer
45 views

Determining a Statistically Significant Result

I feel like this is an easy question if I just knew the right words to Google, but apparently I don't, so I'm sorry if it's been asked before. I'm simply looking to determine how long I need to ...
2
votes
1answer
27 views

What causes low power on hypothesis tests?

I've been doing hypothesis tests to compare two proportions in a product. Although the sample size is supposed to be more than enough based on the population and $\alpha$ (type I error), the power of ...
1
vote
1answer
30 views

Power analysis for t-test instead of ANOVA

Let's say I have an experiment with a 2x2 repeated-measures design. Let's call the four cells A1B1, A1B2, A2B1, A2B2, for the factors A and B, each with two conditions, labeled 1 and 2. I want to ...
0
votes
1answer
23 views

Sample size needed?

I need some help calculating the minimum sample size needed for my upcoming trial. I’m doing higher degree research on eHealth, so I have little knowledge about statistics and sample size calculation. ...
1
vote
0answers
25 views

Power Analysis and Non-Central t Distribution

I'm a bit confused about power analysis and the non-central t-distribution. If I am in a situation where $H_0$ is $\mu = 0$, and $H_1$ is $\mu > 0$, I calculate for what value $\mu$ would have a ...
1
vote
0answers
18 views

Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
1
vote
0answers
17 views

Power calculation for TSSEM or Meta-analytic SEM

I am wondering about power calculations for Cheung's metaSEM/TTSEM. I assume Stage 2 SEM is where having too few studies/small sample sizes becomes problematic. Is there a way to run an a priori ...
1
vote
1answer
16 views

Help with calculating number of animals for a fetal surgery experiment

Rather odd question. I'm a surgical resident who is currently a research fellow in a lab studying fetal surgery. Our plan is to test a glue which we believe will help seal a uterus after we open it to ...
1
vote
2answers
41 views

Statistic metric for groups with different observations

I have this problem and more I read online, more confused I'm getting. So I have 4 (or more) groups with different users (for simplicity I only used small numbers, but I may have 50, 100, 150 and 200 ...
1
vote
2answers
34 views

Is it the case that β+power=1?

In psychology papers that do prospective power analyses (example), one often notices the convention of assuming β values (false negative rates) of 0.20, and power levels of 0.80. In other words (if I ...
1
vote
0answers
42 views

Power of Chi-squared test in R

I am trying to calculate the required sample size for my study. I found the pwr-package to be useful but cannot reproduce the results. What I did: ...
0
votes
1answer
53 views

Is there a relationship between number of covariates and sample size in logistic regression? [duplicate]

Is there some definite relationship between number of covariates and the sample size in logistic regression? (e.g. larger the number of covariates, larger the sample size needed, etc.) Thank you,
7
votes
2answers
252 views

Minimum sample size per cluster in a random effect model

Is there a rational for the number of observations per cluster in a random effect model? I have a sample size of 1,500 with 700 clusters modeled as exchangeable random effect. I have the option to ...
0
votes
0answers
24 views

Power Analysis with Noncentral F distribution

I'm looking to replicate a power analysis for binary data using an MRMC analysis (details found here). The data looks like this, where Score is a binary variable. ...
2
votes
1answer
133 views

How to conduct power analysis for unbalanced one-way ANOVA in R?

I am looking for a way to conduct power analysis for unbalanced one-way ANOVA in R. I found the pwd package; however, it only allows me to calculate power for ...
0
votes
0answers
20 views

Sample size calculation for parametric test, with large effect expected, small numbers. Should I use parametric or non parametric tests?

I am trying to figure out what would be the best way to analyze data from a randomized double blind trial we conducted. We sought to find if two dosages of a drug were effective for a severe symptom ...