Questions tagged [power-analysis]

An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of power (e.g. $80\%$) for a given effect size. In many situation theoretical calculations are intractable so power analysis is done by simulation.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

non central chi square distribution and the non centrality parameter in R [closed]

I want to examinate the power of two sample t test in detecting a shift delta = 2 so i must generate two samples x<-rchisq(10,3) and ...
1
vote
1answer
31 views

Power Analysis and the non central t distribution: what is the non-centrality parameter?

I want to investigate the power of t test in detecting shift so I have a sample $X_1, \dots,X_n \sim N(\mu_x,\sigma)$ and other sample $Y_1, \dots,Y_n \sim N(\mu_y,\sigma)$. My null hypothesis is $H_0:...
-1
votes
0answers
66 views

Power Analysis and the non central t distribution? [closed]

I want to investiagte the power of wilcoxon test in detecting shift .The first scenario is to investigate the power of wilcoxon test of samples stemming from normal distribution.So we have ...
0
votes
0answers
4 views

Adjust alpha in post-hoc-power when bonferroni corrected before?

I was wondering whether I need to adjust my alpha when testing the power of effects for which I have adjusted for multiple testing. This is what I did: Tested the effects of 8 (generalized) linear ...
0
votes
0answers
14 views

How to calculate sample size for blocked-design two-way ANOVA?

Does anyone know of a good online calculator or at least a good paper that details how to calculate the needed sample size for a blocked-design two-way ANOVA experiment? I've seen a ton of references ...
1
vote
0answers
23 views

How to Use Package “simsem” for Sample Size Determination in SEM: Validation Project? [Step-by-step Inquiry] [closed]

To provide context: My model is a latent model; 3 Latent Variables, 3 Indicators for each of two Latent Variables and the last Latent Variable with 4 indicators. Hence, Total of 3 Latent and 10 ...
0
votes
0answers
20 views

What does offsets mean in reference to expectations in R package MultNonParam?

I am trying to calculate sample size for Kruskal-Wallis test using kwsamplesize included with the MultNonParam R pacakge https://cran.r-project.org/web/packages/MultNonParam/MultNonParam.pdf. The ...
0
votes
0answers
13 views

Calculation of the effect size that might have been missed with the available power

I used an extended stratified Cox Proportional Hazards regression model to quantify the rate of an event as a function of exposure to time-varying covariates of interest (binary) and other control ...
0
votes
0answers
30 views

Power of Fisher test vs chi squared

Consider an $m \times n$ contingency table and test of independence. Suppose the expected counts are such that chi-squared test is not recommended (close to zero in some cells), as it becomes pretty ...
5
votes
1answer
98 views

Hypothesis test: numeric vs. ranked

I believed that the most powerful hypothesis test for judging whether a single sample comes from $N(0,1)$ or from $N(1,1)$ uses the average value as test statistics. Thus, I calculate the sample size ...
1
vote
0answers
13 views

Interpreting effect size for binomial GLM (power analysis with simr)

I am doing a power analysis to determine how many sites I need to sample (recording an animal’s presence/absence) to detect a difference between the animal's presence before and after a fire has gone ...
0
votes
0answers
14 views

Simulate data for multilevel & multivariate logistic regression

I want to simulate data for a multilevel logistic regression with several independent variables. I tried doing this with the steps of >>Arnold, B. F., Hogan, D. R., Colford, J. M., & Hubbard,...
0
votes
0answers
20 views

Power analysis for binomial GLM with simr (having trouble changing effect size)

I am trying to determine the impact of bushfire on an animal's presence in a reserve. We already have the pre-fire data (around 200 plots were surveyed and the animal's presence/absence in the plots ...
0
votes
0answers
8 views

Group level t-test when error of summary is known

I'm finding hard to wrap my head around the following. We are planning a study where subjects will be shown different images with varying features. We will measure their response to these images. We ...
1
vote
1answer
31 views

Reagent batch change, sample size, results significancy

We are working on a biological protocol measuring a patient's feature using a blood sample. This protocol has proven to have some variability (CV <= 10%). During this process we sometime need to ...
1
vote
1answer
42 views

Unusual (to me) Phrasing of Power Analysis Objective; Interpretation Requested

I am reading a field biology manual that is directing me to do a power analysis, and I find the phrasing very confusing. It says: Determine sampling requirement to be 80% confident you are within 10% ...
0
votes
0answers
10 views

Confidence vs Power [duplicate]

In the different R packages (like PWR), I have seen examples where power, confidence, and certainty are used, and I have spent time reading how these are the same/different--it's tricky when they're ...
4
votes
1answer
75 views

power analysis for binomial test

I am trying to test whether the sex ratio of some sampled individuals significantly differs from the expected sex ratio of 1. I have n= 64, of which female=34 and male=30. I ran a binomial test: ...
0
votes
0answers
27 views

I am trying to use the simr R package to do an (observed) power analysis for a glmer model

EDIT: Pardon me, I am double-checking things. Some of these numbers may be totally incorrect. This is unexplored territory for me, so I was hoping that somebody could confirm that I am doing this ...
0
votes
0answers
12 views

Formula for sample size for hypothesis test for difference in proportions confusion

For a hypothesis test for difference in proportions: Source A(Some Stanford professor): The sample size for both groups is $$\frac{2\bar{p}(1-\bar{p})(Z_\beta+Z_{\alpha/2})^2}{(p_1-p_2)^2}$$ $$\bar{p}=...
9
votes
1answer
99 views

Does imputation introduce unacceptable bias?

I have recently come to know about imputation techniques, which, in short, "guess" realistic values with which to replace missing values in a dataset. My big issue with this is that we are ...
0
votes
0answers
12 views

Sample Size Calculation - Two Gamma Distributed Mean

I have a similar question to Sample Size Calculation - Two Independent Means, in my case, I am measuring the number of orders our customer make which follows a gamma distribution I believe. How could ...
3
votes
1answer
77 views

How many flips are needed to confirm a coin's intentional bias?

I've made some 'special' coins and I know for a fact that they have a very small but known/designed bias $\epsilon$. I define bias as the probability $(\frac{1}{2} + \epsilon)$ towards heads and $(\...
1
vote
0answers
8 views

How to calculate number of participants and stimuli for experiment to reach significance?

I am running an experiment to determine the best performing of 5 systems, labeled A-E. I plan on running pairwise comparisons, e.g. is A better than B, A vs C, etc.? For the sake of argument, we can ...
1
vote
1answer
31 views

What to do if the data collection company accidentally gathered more data than we need?

This is actually a true story. While planing a study, my work colleague did a power analysis for an intended effect size and a priori power of 80%. It turned out that the required sample size was one ...
0
votes
0answers
40 views

How to calculate power of sample size a posteriori for GLMM

I conducted a study with 3 different groups of participants (n = 25, n = 27, and n = 26) and three phases (baseline, treatment, post-test). I analyzed my data using a mixed-effects binomial logistic ...
1
vote
0answers
33 views

How do I run a power analysis for a 2x2x2 ANOVA?

I have a 2(between) x 2(within) x 2(within) design and would like to calculate the a-priori power needed to detect a three-way interaction using an ANOVA. I've read that this can't be done using power ...
2
votes
1answer
46 views

Should I remove a significant covariate from my mixed-effects model because it's underpowered?

I tried figuring out the answer by reading the comments in this thread but I am still confused. Should I remove a covariate from analysis when it comes as significant but power simulations finds that ...
1
vote
1answer
28 views

Advice on how to answer the “how large of a sample size do we need to show results” question to non-quant people in a business setting

Not sure if this question is too vague, but I'll give it a shot. I've received many questions from non-quantitative colleagues along the lines of "what sample size do we need to trust these ...
2
votes
1answer
34 views

Sample Size Calculation - Two Independent Means

I am trying to design an email test to measure the demand lift obtained from a marketing promotion (treatment) versus no promotion (control). To do so, I want to calculate the per-group sample size ...
1
vote
0answers
95 views

Nested Experimental Design: Intuition about power?

I am planning a nested experimental design, where 1000 participants respond to one policy proposal coming from a 2^3 design (125 participants for each condition A x B x C). Yet, the policy proposal ...
1
vote
0answers
17 views

Power analysis for Ordinal Regression

I am looking for a resource to conduct power analysis for an ordinal regression that includes several predictors (there is no "treatment group"). I found a great simulation method first ...
1
vote
0answers
27 views

How many repetitions to establish difference

(Real-world Context: I often make changes to code, which I suspect will have small impacts on the performance. i.e. improvements that lie within the natural variation of the code's runtime. How many ...
1
vote
0answers
22 views

Power of combined hypotheses test of two separate hypotheses

In a manuscript that I've read, the primary objective is to detect the effect of a treatment. To be able to generalize the statement, two different species (mice and rats) are included in the study. ...
0
votes
0answers
28 views

Sample size to test a three-way interaction

I'm want to run a study with 2 categorical IVs and 1 continuous IV. The DV is continuous. It's my understanding I will need to run a multiple regression to test the effects. How do I determine the ...
0
votes
0answers
53 views

Sample size analysis for retrospective analysis of survey data

I have data that were collected on childhood injuries for 1 year at clinics in certain US states. The data were then aggregated, with weights and strata to be nationally representative. This is ...
0
votes
0answers
15 views

Wilcoxon-Rank-Test / Variance heterogeneity and skewed normal distribution / Power Simulation

I observe results that I cannot explain directly. I have created a simulation study where the data generating process follows a skewed normal distribution where the parameters of mean and variance are ...
0
votes
0answers
15 views

Sample size cutoff for agreement coefficients

New poster here. Would love some help thinking through this. Suppose I am observing individuals in public space, and I am recording the number of occurrences of a target behavior (e.g., looking at a ...
0
votes
0answers
51 views

minimal detectable effect size in linear mixed model

I'm trying to find the minimum detectable effect size (MDES) given my sample, alpha (.05), and desired power (90%) in a linear mixed model setting. I'm using the ...
0
votes
0answers
17 views

Analytical power analysis of a binomial RCT

I have a certain population of users of my free app, with new ones installing every day. I want to run an RCT on them, specifically measuring the impact of some change on their conversion rate (i.e. ...
0
votes
0answers
12 views

Simr: Post hoc power 0% and non signfiicant but effect size 0.3

I set up a glmer model via the package lme4. Using a likelihood ratio tests I compared several models. Two main effects were ...
0
votes
0answers
11 views

How to use Power Spectral Density in a Linear Regression

I’ve computed a vector-valued autocorrelation function $\phi_{xy}(t)$. Because I use an exponential kernel, $\phi$ is pre-windowed. In the discrete world, phi is a 3D matrix (num inputs by num outputs ...
1
vote
0answers
14 views

Power calculation for a one-stage, single arm trial with continuous primary outcome

An early-stage single-arm clinical trial is being planned to evaluate the efficacy and safety of treatment for a bacterial infection. The primary outcome is binary - proportion with >0.5 log10 ...
0
votes
0answers
18 views

power analysis for correlations

Does anyone know of a good method to run a power analysis for correlations whilst taking into consideration the number of trials of the task? I have used the package pwr in R to look at power but it ...
1
vote
1answer
17 views

How do I report the power calculation when I am performing calculations on the whole population?

I am putting together a bid which states: "you should provide information about both your issued and achieved sample sizes, along with appropriate power calculations". The dataset I am using ...
0
votes
0answers
13 views

Calculate power for a cumulative link model, a categorial logistic regresssion in this case with G*Power

I want to calculate the power of a logit model. In R I used the clm function. I know, it is only possible to calculate an approximation. I want to use G*Power for the calculation. But I'm asked to ...
0
votes
0answers
11 views

Estimating sample size in a penalized bernoulli regression

In response to this post Frank Harrell suggests a minimum sample size per sub-group of $n=96$ for penalised Bernoulli regression. I was wondering where he might have got that number from. A paper? or ...
0
votes
0answers
24 views

Analysis plan and Power analysis for nonparametric test with repeated measures

I am conducting a study where each participant goes through 8 trials with a binary outcome variable (i.e., Yes/No). There are two groups, call them A and B. Based on how I am setting up the study, my ...
0
votes
0answers
18 views

Leverage Power Analysis to conclude significance for multiple tests

Here is the case I'm trying to address with hypothesis testing and power analysis. We split(unequally) the population into 2 groups.After the execution of the experiment we measure 2 metrics. 1....
0
votes
0answers
27 views

Calculate minimum detectable effect

I need to calculate analytically the minimum detectable effect when comparing two-sided 2 sample proportions z-test for given sample size (different sample size per group), significance level and Type ...

1
2 3 4 5
14