Questions tagged [power-analysis]

An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of power (e.g. $80\%$) for a given effect size. In many situation theoretical calculations are intractable so power analysis is done by simulation.

Filter by
Sorted by
Tagged with
0
votes
1answer
20 views

Factors affecting SE and power in lmer: how do I have lower SD&CV but higher SE and lower power?

I using simulated data sets to compare the ability to two different experimental designs (A and B) to detect a interaction between two variables (x and y) in determining the observed output (o). The ...
0
votes
1answer
28 views

Sample size calculation resulting in too high of a sample

I am trying to perform a power analysis to determine what sample size to use for an experiment after performing a pilot study. However, the sample sizes are showing values that are way too high and ...
0
votes
1answer
27 views

A power of 0.8 implies a main effect of 2.8

Can anybody please explicate the following statement by Andrew Gelman? If you have 80% power, then the underlying effect size for the main effect is 2.8 standard errors from zero. That is, the z-...
0
votes
0answers
4 views

Simr power for interaction: how to set the effect size for interactions?

I want to calculate power for an interaction (two categorical variables, each with two levels). I'm using the simr package. I'm able to calculate the observed power, but now I would like to modify the ...
0
votes
0answers
6 views

Sample Size for ANOVA with moderation [closed]

How do I calculate the minimal sample size for a one factor between subject experiment if I want to include a moderator?
0
votes
0answers
13 views

Determine necessary sample needed to assess true positives

I am trying to determine what is the sample I need in order to determine if a new method is as effective as an old one. So for example, in three weeks samples were drawn of 234, 229, and 163. These ...
0
votes
1answer
27 views

Is an f-test the correct method for power analysis of my multivariate regression data?

I have been working on solving a problem that requires power analysis to be run in order to determine if the subset of data that we're running analysis on is sufficient before we do the analysis. I ...
1
vote
1answer
22 views

Power analysis of binary predictor in cohort

I'm trying to run a power analysis in Stata, using chi2 (even though I am using logistic regression, I just can't find an option for it and I'm new to power analysis) and it seems I have to choose how ...
1
vote
1answer
35 views

How to interpret sensitivity power analyses?

Journals and reviewers increasingly ask authors to systematically report sensitivity power analyses. I know that a sensitivity power analysis allows you to determine the minimum effect size that the ...
0
votes
1answer
11 views

post hoc power analysis on secondary data using Binary Logistic Regression

I am currently analysing a series of data collected annually. each year varies, however, it is convenience sampled. one year, for example, has a sample of 2427 positive cases, of a known population ...
1
vote
1answer
17 views

In survival analysis, how to calculate power from hazard ratio and number of events given alpha?

If I know number of events and hazard ratio, with given alpha, how do I calculate the power?
1
vote
0answers
10 views

Estimating sample size for treatment on a continuous data with many zeros

I would love to hear peoples guidance/method on approaching this problem. I've have a large continuous reference dataset (n_ref=~50k) with lots of zeros (~95%). I need to estimate the number of ...
1
vote
1answer
34 views

How many examples do I need to show that a change in a feature causes a change in prediction?

not being a statistician I have a bit of a trouble to wrap my head around the following concept. Say I have a dataset $D$ of $N$ data points and a binary classifier (that was fit to some other ...
0
votes
0answers
17 views

How to calculate sample size in a dataset with multiple tests

i need some help. I just cannot figure it out on my own. I got told to find out if we already have enough participants (birds) for this study, so do a power analysis to calculate sample size. ...
0
votes
1answer
18 views

Power calculation & study design for study with rare safety outcome

An RCT is planned to compare a new/repurposed drug to placebo on the basis of mechanistic findings. Limited safety data exist. The primary efficacy outcome requires the recruitment of ~200 ...
0
votes
0answers
19 views

Compute power from a sample, NOT standard error, given power

(Apologies in advance if this is trivial.) Suppose I have a chance to examine a coin for trueness. The way to go about it seems to roll it a few times, estimating if the probability of either face ...
3
votes
1answer
60 views

Model selection with low N?

I've got a study which is kinda messed up by the design...Turns out I ended up with about 50 patients of which 80% have the outcome and 20% don't (binary outcome). I've been turning in my bed for the ...
0
votes
0answers
6 views

Reporting Power analyis

I'm currently on my first attempt at writing up a post-hoc power analysis, but I'm unsure on how this should be reported, for example do I report the power as a percentage or just as it is? How do I ...
0
votes
0answers
17 views

the role of MDE in a/b test analysis

I have a question about the role of the minimum-detectable-difference (MDE) in power analysis before the experiment is run and in post-experiment analysis. To clarify: suppose I am testing the ...
3
votes
1answer
69 views

Sample size selection, known incidence rate distribution vs empirical

I want to estimate the sample size needed to compare binary incidence rate between two populations (based on binary separation to low/high risk groups). The known (previous research) incidence rate ...
2
votes
0answers
21 views

ICC and power for mixed models

I'm simulating data for a mixed model in order to estimate the needed power for a study. I have 3 groups, a control group and two treatment groups. The effect sizes for the two treatment groups are 0....
2
votes
0answers
39 views

Small sample-size, two sample test, power calculation

We have two normally distributed random variables with means $\mu_X$ and $\mu_Y$. We test the null hypothesis $H_0:\mu_X-\mu_Y=\delta=0$, at significance level $\alpha$ in a two-tailed test. We sample ...
0
votes
0answers
15 views

Welch test power

Cross-post from Math StackExchange (https://math.stackexchange.com/questions/3463613/welch-test-power). I know the formulae for the test statistic for the Welch t-test and the degrees of freedom, as ...
0
votes
0answers
9 views

Fitting a Negative Binomial Distribution to small data set

I would like to perform a sample size calculation based on data from a small pilot study. In this experiment, cells are counted in each individual. There will be two different treatment groups which I ...
0
votes
0answers
78 views

2x2x2 factorial design - power analysis

I have a 2 x 2 x 2 factorial within-design. So basically, I would like to study how 3 variables with two levels each affect my dependent variable. What about the a-priori sample size calculation ...
2
votes
1answer
20 views

Power of a statistical test against a compound alternative hypthosis

I was recently posed a homework question. It will be a few days before I can get in-person help from faculty, so I wanted to ask here. The question was: A medical resident doing research wants to ...
1
vote
0answers
16 views

Distribution of noncentrality parameter in noncentral t-distribution

Let $X$ be a random vector $X \sim N(0, \Sigma)$, where $\Sigma = \frac{1}{n} Id$. Find probability $$ P(|T| > a)$$ where $T$ has noncentral t-distribution with noncentrality parameter $\delta = \...
1
vote
0answers
19 views

Power function versus p-value for a normal test of means

In my lecture notes, the power function under a test with rejection region $R$ is defined as: $ \beta(\theta) = P_{\theta} ((X_1 ,..., X_n) \in R) $ and the p-value is defined as $ p = inf(\alpha $ : ...
4
votes
1answer
74 views

Performing power analysis with clustered standard errors, interaction effects, and covariates using pilot data

Let's say I run an experiment where participants play a behavioral game with someone that is either their same gender, or not their own gender. The primary criteria of interest is a pre and post ...
3
votes
0answers
89 views

How can I calculate the power of my analysis with binary response data?

I have three groups with ~ 15 individuals per group. Group 1 gets a treatment Group 2 is a control in the same area Group 3 is a further control in a different area The response is binary, 1 or ...
0
votes
0answers
15 views

Simulating correlated discrete choice data for power calculations

Discrete Choice Data While discrete choice experiments take many forms, one common form is for $N$ participants to choose the most preferred option from an option set comprised of $W$ options for $K$ ...
2
votes
0answers
31 views

How to do power analysis for Fisher exact test in tables larger than 2x2? [duplicate]

I have 4x2 contingency tables and would like to perform Fisher's exact test on them. R stats package(https://www.rdocumentation.org/packages/stats/versions/3.6.1/topics/fisher.test) supports 4x2 test ...
1
vote
0answers
31 views

Does my evidence against the $H_0$ increase with increasing power but fixed $\alpha$? (e.g. with increasing sample size)

Let's say I test $$H_0: \theta \in [-1,0] \quad vs. \quad H_1: \theta \in (0,1]$$ where $\Theta = [-1,1]$ and let's say I get a p-value $ p\leq \alpha$. My question is whether the evidence this ...
0
votes
0answers
58 views

simR: How to interpret output of powerSim when using compare() or fcompare()

I've been using simR to calculate the power of linear mixed effects regression (lmer) models, given a set of pilot data. For example, when using the built-in ...
0
votes
0answers
30 views

Plotting power curve with power on y-axis, effect size on x-axis, sample size held constant

I'm performing a multivariate power analysis on episode-level medical data to estimate the minimum average cost reduction we can detect with an 80% positive identification rate (i.e., power = 0.80) ...
0
votes
0answers
8 views

How to estimate power for individual associations in a regression?

I am conducting a study looking at measures of sensory sensitivity, disgust sensitivity, and 3 temperament traits as predictors of severity of an illness. Thus my dependent variable is severity of ...
3
votes
0answers
46 views

Power and sample size calculations for unpaired ROC curves

I would like to perform power and sample size calculations for comparison of unpaired receiver-operating characteristic (ROC) curves. I have tried using the power.roc.test function from pROC package ...
1
vote
0answers
18 views

Fit one model or several models to maximize power for significance testing of slopes in a factorial design?

Problem I need to perform a power analysis for a set of statistical tests. Each test is about a regression slope computed in a cell of a factorial experiment design. To minimize the required sample ...
0
votes
1answer
19 views

Why is expected group difference necessary in power analysis?

Why do we need the expected group statistic (i.e. mean) difference when calculating the sample size? The point of power analysis is to find the right sample size so that I can detect a difference. ...
2
votes
0answers
94 views

R: Power analysis for a 2 by 2 within-within ANOVA interaction effect [pwr::pwr.f2.test()]

Background I would like to perform a power analysis for the interaction effect in a 2 by 2 within-within ANOVA design. I would like to crosscheck that I am performing this correctly using the ...
0
votes
1answer
10 views

how to calculate pre and post measures degrees of freedom across participants

I have 10 participant and I want to measure if there is a correlation between their main measure and a clinical condition. I want to know how to calculate the degree of freedom pre and post measures ...
1
vote
1answer
39 views

Meaning of Power Analysis function parameters for SSizeLogisticBin when A/B test in R

I did recently an A/B testing course in R from DataCamp and I could not understand the parameter B from the following function from the package powerMediation ...
0
votes
0answers
21 views

How many experiments to run (sample-size) if I know I am going to feed them to a non-parametric regression?

I have 2 input variables, $X_1$ and $X_2$ that affect output variable $Y$. I can run experiments where I modify the inputs and measure what happens to the output. Now, if $X_1$ and $X_2$ were binary, ...
0
votes
1answer
156 views

Simulation in R of data based on Cox proportional-hazards model for power analysis

I would like to fit data based on Cox proportional-hazards model and then simulate new data based on a fitted model. For instance, provided that the fit is ...
0
votes
2answers
37 views

The calculation method of post-hoc power in t-test without welch's correction

【About this thread】: This thread is divided from the following threads: What is the post-hoc power in my experiment? How to calculate this? I cut out for when Welch's correction is not ...
0
votes
2answers
158 views

What is the post-hoc power in my experiment? How to calculate this?

The following articles are reprinte of #3375492 of math.stackexchange.com. It was recommended to ask this community at math.stackexchange.com. My motivations I often see the claims that post-hoc ...
0
votes
0answers
5 views

Do I need to run an “a priori” power analysis before doing a content analysis?

I am planning to do a content analysis of social media posts from different groups, coding categories with categorical (binary) variables (0 = not present, 1 = present). I want to compare group ...
0
votes
0answers
12 views

Post hoc power analysis with multicategorical antecedent

I need to report actual power with a post hoc power analysis. My IV is multicategorical (control, condition 1, condition 2). I examined direct effects on two outcomes, indirect effects via 10 ...
1
vote
0answers
79 views

PowerSim function (simr package) indicates 100% power for small sample dataset, regardless of effect size, alpha level, or if fixed/random slopes

I am using the simr package to do power analyses for lmer multilevel models I have run, to determine the power of a pilot dataset for future research. The dataset consists of 46 subjects with ...
0
votes
1answer
26 views

Power analysis in R - Sample size

I have a question regarding power analysis in R using the following function: library(pwr2) pwr.2way(a=2, b=8, alpha=0.01, size.A=12, size.B=12, f.A=0.47, f.B=0.4) ...

1 2 3 4 5 13