Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [power-analysis]

An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of power (e.g. $80\%$) for a given effect size. In many situation theoretical calculations are intractable so power analysis is done by simulation.

0
votes
0answers
2 views

Signal detection theory: one-interval design (= yes-no task) and 2AFC design, test power and sample size

my research group and I read some more about signal detection theory and we were not able to find answers concerning the following questions. Therefore, we would appreciate your help very much! 1) We ...
0
votes
0answers
7 views

Experimental Design Fixed Effect Minimal Detectable Difference

How does one determine the minimal detectable difference for a fixed factor/interaction in a general balanced experimental design? For example, I would like to be able to recreate Appendix 12 of ...
2
votes
2answers
74 views

Can bootstrapping improve the test power, when the sample size is small? [on hold]

When the sample is small and study is under-powered, can bootstrapping be used to generate data and increase the sample size (and improve the test power)? Is there any way that bootstrapping can be ...
1
vote
1answer
15 views

Power analysis: number of participants or number of data points?

This almost answered my question below, but not quite. I want to calculate the sample size required in order to reach a certain level of a priori statistical power in my experiment. My question is ...
0
votes
0answers
8 views

Estimate sample size in respect to percent of difference between two means

I am trying to find a way to estimate (even in a crude manner) what sample size I need to detect a mean difference of say 5% between two independent groups. Problem is, I do not know sigma nor there ...
0
votes
0answers
13 views

How do I calculate the statistical reliability of a random sampling of features?

I have a dataset of 6 million features from which I've trained a binary classification model. It identified 32 features that can accurately predict 98% of samples (ie if I train a model with only ...
0
votes
0answers
25 views

Calculating power for balanced incomplete block design

My experiment has a factorial 2x2 design: 2 factors are manipulated at levels 'high' and 'low'. We thus get 4 conditions: Condition 1: Factor A high, Factor B high Condition 2: Factor A high, Factor ...
0
votes
0answers
10 views

How to calculate statistical power in a clustering problem?

I'm considering following question. I implement a algorithm to cluster my data and the result will assign each cluster to a small cluster or big cluster according to its size. After that, I will get ...
0
votes
0answers
21 views

(Sample Size calculation) G*Power analysis: ANCOVA

I wrote a similar post just before, but I realized that I put wrong information there, hence I deleted it, and have re-written here I am trying to conduct a power analysis on a hierarchical ...
1
vote
1answer
31 views

Power analysis for same model as “ANOVA” vs as “multiple regression” yields different results

I have seen posts that said ANOVA and multiple regression are theoretically the same. But if this is really the case, does anyone know why the G*Power (Linear multiple regression vs ANOVA) gives ...
0
votes
0answers
14 views

Minimum sample size calculation a posteriori

I am running a number of experiments where I obtain a large number of observations with one variable (yielding two sets of values; many more than would be needed for a hypothesis test). I want to test ...
0
votes
0answers
7 views

Sample Size: G*Power for hierarchical multiple regression with a three-way interaction?

I have been struggling quite a lot with finding a way to conduct a power analysis for a hierarchical regression with a three-way interaction. I will put 2 control variables in Step 1, 3 IVs in Step 2 ...
1
vote
1answer
24 views

Sample size needed to prove equal proportions

We are trying to prove that three groups have the same proportion of some effect. We are struggling with formulating the null hypothesis, and whether or not the sample size is big enough. The ...
1
vote
1answer
36 views

Power of paired t-test: R vs. SAS

If we are trying to find the power of a paired t-test, SAS offers this template (among others) in PROC POWER: ...
0
votes
0answers
37 views

Sample size calculation in repeated measures design for linear mixed models

Does anyone know a method how to calculate the required sample size in a repeated measures design that will be analysed using linear mixed modelling? The design looks as follows:I have two groups (...
1
vote
0answers
24 views

Is 1/5 of studies that use 80% power calculations wrong?

I'm just learning about power calculations. As I understood, the most commonly used power is 80%, meaning that if there is an effect, the null hypothesis is rejected in 80% of the cases. However, I'm ...
1
vote
1answer
26 views

Power analysis for multilevel model with two fixed and two random (crossed) effects: how to define sample size with simr package [closed]

I want to define the sample size for a repeated measurement experiment. Every subject will indicate her/his behavior on a 1-7 scale for several stimuli (=behavior). I expect that personality and ...
0
votes
0answers
13 views

Power Analysis for Case Control Regression?

If we conduct a case control study and plan to analyze the results with multiple linear regression, which power analysis should be conducted to select sample size? There are separate calculators for ...
3
votes
1answer
42 views

Determining a Statistically Significant Result

I feel like this is an easy question if I just knew the right words to Google, but apparently I don't, so I'm sorry if it's been asked before. I'm simply looking to determine how long I need to ...
2
votes
1answer
24 views

What causes low power on hypothesis tests?

I've been doing hypothesis tests to compare two proportions in a product. Although the sample size is supposed to be more than enough based on the population and $\alpha$ (type I error), the power of ...
1
vote
1answer
26 views

Power analysis for t-test instead of ANOVA

Let's say I have an experiment with a 2x2 repeated-measures design. Let's call the four cells A1B1, A1B2, A2B1, A2B2, for the factors A and B, each with two conditions, labeled 1 and 2. I want to ...
0
votes
1answer
19 views

Sample size needed?

I need some help calculating the minimum sample size needed for my upcoming trial. I’m doing higher degree research on eHealth, so I have little knowledge about statistics and sample size calculation. ...
1
vote
0answers
14 views

Power Analysis and Non-Central t Distribution

I'm a bit confused about power analysis and the non-central t-distribution. If I am in a situation where $H_0$ is $\mu = 0$, and $H_1$ is $\mu > 0$, I calculate for what value $\mu$ would have a ...
1
vote
0answers
15 views

Does separation of data matter in Bayesian power calculations with logistic regression models?

I know that, when performing power calculation based on logistic regression model, under the regular frequentist approach, power calculations become unstable if it is based on the model that causes ...
1
vote
0answers
15 views

Power calculation for TSSEM or Meta-analytic SEM

I am wondering about power calculations for Cheung's metaSEM/TTSEM. I assume Stage 2 SEM is where having too few studies/small sample sizes becomes problematic. Is there a way to run an a priori ...
1
vote
1answer
16 views

Help with calculating number of animals for a fetal surgery experiment

Rather odd question. I'm a surgical resident who is currently a research fellow in a lab studying fetal surgery. Our plan is to test a glue which we believe will help seal a uterus after we open it to ...
1
vote
2answers
38 views

Statistic metric for groups with different observations

I have this problem and more I read online, more confused I'm getting. So I have 4 (or more) groups with different users (for simplicity I only used small numbers, but I may have 50, 100, 150 and 200 ...
1
vote
2answers
33 views

Is it the case that β+power=1?

In psychology papers that do prospective power analyses (example), one often notices the convention of assuming β values (false negative rates) of 0.20, and power levels of 0.80. In other words (if I ...
1
vote
0answers
37 views

Power of Chi-squared test in R

I am trying to calculate the required sample size for my study. I found the pwr-package to be useful but cannot reproduce the results. What I did: ...
0
votes
1answer
44 views

Is there a relationship between number of covariates and sample size in logistic regression? [duplicate]

Is there some definite relationship between number of covariates and the sample size in logistic regression? (e.g. larger the number of covariates, larger the sample size needed, etc.) Thank you,
7
votes
2answers
162 views

Minimum sample size per cluster in a random effect model

Is there a rational for the number of observations per cluster in a random effect model? I have a sample size of 1,500 with 700 clusters modeled as exchangeable random effect. I have the option to ...
0
votes
0answers
18 views

Power Analysis with Noncentral F distribution

I'm looking to replicate a power analysis for binary data using an MRMC analysis (details found here). The data looks like this, where Score is a binary variable. ...
2
votes
1answer
54 views

How to conduct power analysis for unbalanced one-way ANOVA in R?

I am looking for a way to conduct power analysis for unbalanced one-way ANOVA in R. I found the pwd package; however, it only allows me to calculate power for ...
0
votes
0answers
18 views

Sample size calculation for parametric test, with large effect expected, small numbers. Should I use parametric or non parametric tests?

I am trying to figure out what would be the best way to analyze data from a randomized double blind trial we conducted. We sought to find if two dosages of a drug were effective for a severe symptom ...
0
votes
1answer
32 views

G*Power and simulation produce very different sample size estimates for a Poisson regression

I've been attempting to conduct a sample size calculation for a Poisson regression. G*Power produced a sample size of 472. Parameters for G*Power Tails = 2 Exp(B1) = 1.233 alpha = 0.05 Power = 0....
2
votes
2answers
82 views

How to calculate the effect size for a t-test?

The pwr package includes the function pwr.t.test, which relates the following four quantities: sample size effect size ...
0
votes
1answer
43 views

Minimum sample size to trust univariate cox regression

I have performed a univariate Cox proportional hazards regression for an admittedly miniscule group of 25 samples, and consequently received a certain p-value (significant) and an associated ...
0
votes
0answers
12 views

Should I test for sphericity and calculate statistical power before running and modelling any forecasting model?

I'm trying to model a regression for one dependent and five independent macroeconomic variables. I'm new to statistics but I've read lots of text books and articles about SEM, cointegration, ols, URT, ...
2
votes
1answer
23 views

calculate sample size, differences between software

Currently I'm doing a power calculation using the pwr package. I thought it should be very simple, but I'm not sure if I'm doing it right. I use R to calculate the sample size. I checked my results ...
4
votes
0answers
54 views

What are appropriate statistic tests for top-down and bottom-up knockout experimental designs?

This concernes a two staged experiment on cell toxicality. A: Aim is to find the lowest concentration of the toxic agent 'CTA', that kills the target cells with at least 99% security. Use of lowest ...
0
votes
0answers
4 views

Clustered or blocked? Running power analysis for DIM

I am working with the following design: -80 districts, each district paired using matching and half assigned to T1 and the other to C1 -All sub-districts within a district (let's say 10) receive T--> ...
0
votes
0answers
22 views

power analysis for interaction of three within-subject factors

I am trying to find an example of power analysis for an interaction effect of three within-subject factors (each factor has 2 levels). I found only an example for an r code simulation with 2 factors, ...
3
votes
2answers
68 views

Power analysis for logistic regression with dummy independent variables [duplicate]

Per reviewer request, I need to do power analysis for a logistic regression model with multiple dummy variables. I have four groups: Control, (Treatment) A, B, and C. The hypothesis is that group A ...
2
votes
1answer
41 views

Sample size calculation for Kendall's Tau in reproducibility study

Can anyone help with justifying (or rubbishing) a couple of aspects of a sample size calculation for a grant proposal. A pilot study gave significant results for the correlation of a continuous (non-...
0
votes
0answers
37 views

Calculating Sample Size for Cluster Randomized Trials with Person-level or Cluster-Level Outcomes

I have been looking for a formula to calculate the estimated sample size for a two-level cluster-randomized controlled trial with cluster-level outcome data. To my reading, several authors do not make ...
0
votes
0answers
28 views

Can I break up observational window in order to increase sample size?

I want to compare a tasks completed in the morning to tasks completed in the evening. The participants are given 4 hours to complete the tasks. I will do a t-test to compare the effectiveness of the ...
0
votes
0answers
9 views

Measurement theory and sample size calculation for multivariate testing

Let $\mathbf{Y}$ be a vector of independent, normally-distributed random variables. Let $S_1$, $S_2$ and $S_3$ be three non-overlapping samples of sizes $N_1$, $N_2$ and $N_3$, respectively. Let $M_A$ ...
1
vote
0answers
18 views

Power analysis for contingency table

Following a cardiac surgery a patient will be classified into one of four categories: Nothing, Mild, Moderate, Severe. There is a new test which, when performed after the surgery, claims to be able to ...
0
votes
1answer
30 views

How to compensate for different statistical power in multiple comparisons? [closed]

Imagine we are performing 10 comparisons on a single variable of choice. The 10 respective experiments have different statistical powers. Now imagine that the least powerful and the most powerful ...
0
votes
0answers
33 views

Sample size calculation: difference between 'linear multiple regression test' & 'linear bivariate regression' test G*Power 3.1

I am trying to calculate a sample size using G*Power 3.1 for a linear model with multiple independent variables, but only two "primary" predictors of interest plus an interaction term between them. ...