Questions tagged [power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

532 questions
Filter by
Sorted by
Tagged with
7 views

rule of thumb for generalizing observations to population

I recently had a question from a co-worker, “How big a sample would you need to feel comfortable communicating figures?” My answer was two part: 1) if we’re trying to show something made a ...
9 views

Can pooled standard deviation be used to determine sample size?

I would like to use Minitab statistical software to determine a minimum sample size for precision (repeatability) testing. From casual observation, we know our allowable error is much wider than the ...
13 views

Power function on a calculator (=normalcdf)

I am preparing myself for a statistics exam and got stuck on a question. Namely, when asked to calculate the power 1-beta of a statistical test I see that in solutions to some exercises they use 1-...
16 views

How to calculate sample size in a dataset with multiple tests

i need some help. I just cannot figure it out on my own. I got told to find out if we already have enough participants (birds) for this study, so do a power analysis to calculate sample size. ...
17 views

Power calculation & study design for study with rare safety outcome

An RCT is planned to compare a new/repurposed drug to placebo on the basis of mechanistic findings. Limited safety data exist. The primary efficacy outcome requires the recruitment of ~200 ...
14 views

required sample size for establishing equivalence of a gradient boosting model on different population

I have a trained Gradient boosting Trees (regression) model with a given R2 metric (obtained via cross validation) Now I want to verify that the same model is valid for a very different population. Is ...
22 views

How can one define the power of a randomized clinical trial?

The statistical power of a study is a measure of the ability of a randomized clinical trial (RCT) to detect a difference statistically significant of the treatment effect in the treatment arms. How ...
19 views

Does the statistical power for testing regression coefficients depend on the number of model parameters?

Given a generalized linear model (GLM), does the number of regression coefficients affect the power to detect a significant effect size for an individual regression coefficient i.e. as more variables ...
9 views

Power calculation for multinomial (polytomous) logistic regression

I am struggling to find a software or simply a way to do a power analysis for a multinomial regression. I have 4 outcome (dependent) groups that are nominal and non-ordered. My main exposure (...
31 views

Multiple comparison adjustment with multiple outcomes, measured over time

I am trying to work out the most appropriate way to adjust for multiple comparisons. I have 20 biomarkers which have been measured at 4 timepoints (baseline & three unevenly spaced follow-ups). I ...
7 views

Power Law with log of DV against log of CDF

I am trying to find a way to proof that my DV follows a distribution close to power law. How is this possible with SPSS? Is there a way to plot the log of the DV against the log of the CDF? I am ...
20 views

Is a best critical region unique?

For testing a simple hypothesis $H_0:\theta=\theta_0$ against another simple hypothesis $H_1: \theta=\theta_1$, a best critical region or a most powerful test of size (aka, significance level) $\alpha$...
19 views

Power of a statistical test against a compound alternative hypthosis

I was recently posed a homework question. It will be a few days before I can get in-person help from faculty, so I wanted to ask here. The question was: A medical resident doing research wants to ...
25 views

How can I calculate the power of my analysis with binary response data?

I have three groups with ~ 15 individuals per group. Group 1 gets a treatment Group 2 is a control in the same area Group 3 is a further control in a different area The response is binary, 1 or ...
31 views

Does my evidence against the $H_0$ increase with increasing power but fixed $\alpha$? (e.g. with increasing sample size)

Let's say I test $$H_0: \theta \in [-1,0] \quad vs. \quad H_1: \theta \in (0,1]$$ where $\Theta = [-1,1]$ and let's say I get a p-value $p\leq \alpha$. My question is whether the evidence this ...
158 views

Comparison of statistical tests exploring co-dependence of two binary variables

Suppose we observe data $(X_i,Y_i)_{i=1,...,n}$ on two binary variables: $X\in\{0,1\}$ and $Y\in\{0,1\}$. We would like to test if $X$ and $Y$ are co-dependent (related). Standard suggestions in ...
386 views

Difference/relationship between power and significance

What is the conceptual difference between statistical power and statistical significance? Are they the same? Is one used for the other?
8 views

How to estimate power for individual associations in a regression?

I am conducting a study looking at measures of sensory sensitivity, disgust sensitivity, and 3 temperament traits as predictors of severity of an illness. Thus my dependent variable is severity of ...
10 views

how to calculate pre and post measures degrees of freedom across participants

I have 10 participant and I want to measure if there is a correlation between their main measure and a clinical condition. I want to know how to calculate the degree of freedom pre and post measures ...
21 views

How many experiments to run (sample-size) if I know I am going to feed them to a non-parametric regression?

I have 2 input variables, $X_1$ and $X_2$ that affect output variable $Y$. I can run experiments where I modify the inputs and measure what happens to the output. Now, if $X_1$ and $X_2$ were binary, ...
16 views

Minimum Detectable Effect (MDE) for churn rates

I am new to experimentation and could use some direction. The Minimum detectable effect (MDE) is a calculation that estimates the smallest improvement you’re willing to be able to detect in your ...
16 views

Does anyone know how to conduct power analysis and estimate sample size for a mixed model cross-level interaction?

I am planning for a rather simple mixed design study. I have one within-subject factor (2-level, repeatedly measure/treatment twice) and one continuous between-subject factor. Now I am interested in ...
157 views

Different results for simulated and closed form distribution of T-statistic under the alternative hypothesis

While running some simulations to get a better grasp of the concept of statistical power I stumbled upon an unexpected result. I was trying to simulate the sampling distribution of test statistic $T$ ...
55 views

ROC-style curves for calculating sample size, power, alpha, and effect size

I found an awesome R package called pwr that does all sorts of calculations about sample sizes, power, effect sizes, and so on, and I've been playing. I have a ...
42 views

How can I determine whether my Cox regression analysis have enough power to conclude an outcome

I have 159 tumor patients, of which I know several parameters: age, sex, stage, ... and x. X is a continuous covariate corresponding to the percentage of tumor cells currently dividing (biological ...
18 views

Pooling F-values? Effect sizes and sample size estimations

Im struggling with the power in general.. I need assistance with how to calculate sample size with confidence intervals as to be able to conclude that the sample of the present study is within XX % ...
15 views

Why is likelihood ratio two-tailed and UMP?

How can likelihood ratio ~ chi-squared be a uniformly most powerful test when the hypothesis tested are two-tailed? I thought UMP was only one-tailed.
26 views

R power.prop.test for two samples with different proportions

I am trying to modify power.prop.test is R to adapt for cases where n1 and n2 (two sample sizes) are not equal. That is, they can not be given by one constant n. The equation in R for n1 = n2 = n ...
843 views

Why is power of a hypothesis test a concern when we can bootstrap any representative sample to make n approach infinity?

Why do we care about the power of a hypothesis test if we no longer live in an age where computers are slow and it's too costly to bootstrap/do a permutation test on anything which is also non-...
29 views

Nonlinear Wald Test

Say we have a bivariate distribution $~~~~~f(x,y|\theta=\{a,b,t,c\}).$ An event of interest is when ab=c, that is, $~~~~~H_0: g(\theta)=c-ab=0$ versus $H_1: g(\theta)=c-ab \neq 0.$ I attempt to ...
31 views

Power Calculations for 3 arm clustered randomised trial

I need to do power calculations for a 3 arm trial (to determine minimum detectable effect size, given the alpha, power, sample size etc). I found the R package Three.Arm.Trials, however I was ...
38 views

Why does G*power calculate the noncentrality parameter in this way?(different from formula)

I'm calculating the power of some F-family tests and I found the key to the power was the noncentrality parameter (ncp). So I'm trying to calculate the ncp $\lambda$. However, I got different results ...
36 views

Should a statistical test be performed if power is very low?

I performed some partitioning of data set (genes) into two lists, resulting in two enrichment lists. I wanted to compare them with Fisher's test, but from before I know that similarly looking results ...
38 views

Statistically significant definition of power

After a not-quick literature research I noticed that there are two types od definition for what power is: One is: it's the probability of not making type II error' (A) The other is: it is the ...
13 views

GLM - sample size estimation for given power and effect size and data set

I am working with R's car package highway1 and calculating a general linear model with rate as the dependant var (metric) and acpt as the independant var (metric). Additionally i used htype (...
65 views

Binary Logit: A statistically insignificant predictor become significant after exclude some observation

I am currently trying to explore the determinants of management turnover using binary logit. In robustness check, I have added more control variable (Industry) to see whether the same results hold. ...
43 views

Power & Effect Size for Multiple Regression - HELP

2 IV's: Pregnant/Non Pregnant & Physically Active/Not Physically Active 1 DV: Cognitive Functioning (Inhibition - SSRT scores) AIM: Investigate whether there is a difference in physical activity ...
414 views

Power calculation for two-sample Welch's t test

The R function power.t.test does power calculations (outputs power, sample size, effect size, or whichever parameter you leave out) for t-tests, but only has a ...
10 views

Implications of adding extra “control” conditions to an ANOVA?

I'm planning a 2x3 ANOVA, all between subjects. I'm expecting a weak main effect for A...a main effect for B where one condition is different than the other two...and then an interaction effect where ...
54 views

Using the $Z$-test instead of the binomial test for small $n$ - effect on power and Type I error

Setup Let $M \sim \text{Bin}(p,n)$. Let us then state the following two hypotheses: \begin{align} H_0: p = 0.5 \text{ vs } H_1: p > 0.5 \end{align} To test these hypotheses, I could use the ...
14 views

Using power and significance to calculate sample size in a one-tailed test?

I am trying to determine the sample size needed to demonstrate that the lower boundary of a 95% CI is not lower than the population mean of another, with a power of .9. Right now the formula I'm using ...
126 views

Importance sampling estimation of power function

Problem Suppose we are given $\text{Poisson}(\theta)$ model, and the null hypothesis is as follows: $$H_0 : \theta = 0.1 \ \ \text{vs}. \ \ H_1 : \theta < 0.1$$ Suppose we take sample of \$n=...
64 views

Power of a Poisson hypothesis test

A textbook of mine has online worked solutions, and the answer in the back of the book does not match the answer in the online solutions - they're not even close. My book says that the power of a ...
140 views

simr power simulation testing interaction term for 2 factors

I have a model with two factors: a (4 levels) and b (3 levels). Each participant receives two problems (id as a random effect). I want to estimate sample size based on pilot data to give me ...
25 views

Need help determining sample size for an AB test with two metrics

I need to run an experiment with customer support agents who are rated on two metrics by the customer: average issue resolution rate (proportion of issues resolved) and average star rating (1-5). ...
200 views

Explanation of Advantages of Balanced Designs with Statistical Theory

I wanted to know what the difference between balanced and unbalanced design. Firstly, here are two advantages of balanced designs: ...
26 views

Does a more powerful test mechanically lead to more type III errors then?

"Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. Say that a treatment increases some ...
22 views

Power of Tests for Proportions: One Proportion vs. Two Proportions

I am trying to calculate the required sample size for a trial, where my outcome is proportions (for each subject I am measuring if the subject had an event or not). I tried two ways and got different ...