Questions tagged [power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

rule of thumb for generalizing observations to population

I recently had a question from a co-worker, “How big a sample would you need to feel comfortable communicating figures?” My answer was two part: 1) if we’re trying to show something made a ...
1
vote
0answers
9 views

Can pooled standard deviation be used to determine sample size?

I would like to use Minitab statistical software to determine a minimum sample size for precision (repeatability) testing. From casual observation, we know our allowable error is much wider than the ...
0
votes
0answers
13 views

Power function on a calculator (=normalcdf)

I am preparing myself for a statistics exam and got stuck on a question. Namely, when asked to calculate the power 1-beta of a statistical test I see that in solutions to some exercises they use 1-...
0
votes
0answers
16 views

How to calculate sample size in a dataset with multiple tests

i need some help. I just cannot figure it out on my own. I got told to find out if we already have enough participants (birds) for this study, so do a power analysis to calculate sample size. ...
0
votes
1answer
17 views

Power calculation & study design for study with rare safety outcome

An RCT is planned to compare a new/repurposed drug to placebo on the basis of mechanistic findings. Limited safety data exist. The primary efficacy outcome requires the recruitment of ~200 ...
0
votes
0answers
14 views

required sample size for establishing equivalence of a gradient boosting model on different population

I have a trained Gradient boosting Trees (regression) model with a given R2 metric (obtained via cross validation) Now I want to verify that the same model is valid for a very different population. Is ...
0
votes
1answer
22 views

How can one define the power of a randomized clinical trial?

The statistical power of a study is a measure of the ability of a randomized clinical trial (RCT) to detect a difference statistically significant of the treatment effect in the treatment arms. How ...
1
vote
0answers
19 views

Does the statistical power for testing regression coefficients depend on the number of model parameters?

Given a generalized linear model (GLM), does the number of regression coefficients affect the power to detect a significant effect size for an individual regression coefficient i.e. as more variables ...
0
votes
0answers
9 views

Power calculation for multinomial (polytomous) logistic regression

I am struggling to find a software or simply a way to do a power analysis for a multinomial regression. I have 4 outcome (dependent) groups that are nominal and non-ordered. My main exposure (...
0
votes
2answers
31 views

Multiple comparison adjustment with multiple outcomes, measured over time

I am trying to work out the most appropriate way to adjust for multiple comparisons. I have 20 biomarkers which have been measured at 4 timepoints (baseline & three unevenly spaced follow-ups). I ...
0
votes
0answers
7 views

Power Law with log of DV against log of CDF

I am trying to find a way to proof that my DV follows a distribution close to power law. How is this possible with SPSS? Is there a way to plot the log of the DV against the log of the CDF? I am ...
1
vote
0answers
20 views

Is a best critical region unique?

For testing a simple hypothesis $H_0:\theta=\theta_0$ against another simple hypothesis $H_1: \theta=\theta_1$, a best critical region or a most powerful test of size (aka, significance level) $\alpha$...
2
votes
1answer
19 views

Power of a statistical test against a compound alternative hypthosis

I was recently posed a homework question. It will be a few days before I can get in-person help from faculty, so I wanted to ask here. The question was: A medical resident doing research wants to ...
3
votes
0answers
25 views

How can I calculate the power of my analysis with binary response data?

I have three groups with ~ 15 individuals per group. Group 1 gets a treatment Group 2 is a control in the same area Group 3 is a further control in a different area The response is binary, 1 or ...
1
vote
0answers
31 views

Does my evidence against the $H_0$ increase with increasing power but fixed $\alpha$? (e.g. with increasing sample size)

Let's say I test $$H_0: \theta \in [-1,0] \quad vs. \quad H_1: \theta \in (0,1]$$ where $\Theta = [-1,1]$ and let's say I get a p-value $ p\leq \alpha$. My question is whether the evidence this ...
13
votes
1answer
158 views

Comparison of statistical tests exploring co-dependence of two binary variables

Suppose we observe data $(X_i,Y_i)_{i=1,...,n}$ on two binary variables: $X\in\{0,1\}$ and $Y\in\{0,1\}$. We would like to test if $X$ and $Y$ are co-dependent (related). Standard suggestions in ...
5
votes
4answers
386 views

Difference/relationship between power and significance

What is the conceptual difference between statistical power and statistical significance? Are they the same? Is one used for the other?
0
votes
0answers
8 views

How to estimate power for individual associations in a regression?

I am conducting a study looking at measures of sensory sensitivity, disgust sensitivity, and 3 temperament traits as predictors of severity of an illness. Thus my dependent variable is severity of ...
0
votes
1answer
10 views

how to calculate pre and post measures degrees of freedom across participants

I have 10 participant and I want to measure if there is a correlation between their main measure and a clinical condition. I want to know how to calculate the degree of freedom pre and post measures ...
0
votes
0answers
21 views

How many experiments to run (sample-size) if I know I am going to feed them to a non-parametric regression?

I have 2 input variables, $X_1$ and $X_2$ that affect output variable $Y$. I can run experiments where I modify the inputs and measure what happens to the output. Now, if $X_1$ and $X_2$ were binary, ...
0
votes
0answers
16 views

Minimum Detectable Effect (MDE) for churn rates

I am new to experimentation and could use some direction. The Minimum detectable effect (MDE) is a calculation that estimates the smallest improvement you’re willing to be able to detect in your ...
0
votes
1answer
16 views

Does anyone know how to conduct power analysis and estimate sample size for a mixed model cross-level interaction?

I am planning for a rather simple mixed design study. I have one within-subject factor (2-level, repeatedly measure/treatment twice) and one continuous between-subject factor. Now I am interested in ...
1
vote
0answers
157 views

Different results for simulated and closed form distribution of T-statistic under the alternative hypothesis

While running some simulations to get a better grasp of the concept of statistical power I stumbled upon an unexpected result. I was trying to simulate the sampling distribution of test statistic $T$ ...
2
votes
1answer
55 views

ROC-style curves for calculating sample size, power, alpha, and effect size

I found an awesome R package called pwr that does all sorts of calculations about sample sizes, power, effect sizes, and so on, and I've been playing. I have a ...
1
vote
0answers
42 views

How can I determine whether my Cox regression analysis have enough power to conclude an outcome

I have 159 tumor patients, of which I know several parameters: age, sex, stage, ... and x. X is a continuous covariate corresponding to the percentage of tumor cells currently dividing (biological ...
0
votes
0answers
18 views

Pooling F-values? Effect sizes and sample size estimations

Im struggling with the power in general.. I need assistance with how to calculate sample size with confidence intervals as to be able to conclude that the sample of the present study is within XX % ...
0
votes
0answers
15 views

Why is likelihood ratio two-tailed and UMP?

How can likelihood ratio ~ chi-squared be a uniformly most powerful test when the hypothesis tested are two-tailed? I thought UMP was only one-tailed.
0
votes
0answers
26 views

R power.prop.test for two samples with different proportions

I am trying to modify power.prop.test is R to adapt for cases where n1 and n2 (two sample sizes) are not equal. That is, they can not be given by one constant n. The equation in R for n1 = n2 = n ...
7
votes
1answer
843 views

Why is power of a hypothesis test a concern when we can bootstrap any representative sample to make n approach infinity?

Why do we care about the power of a hypothesis test if we no longer live in an age where computers are slow and it's too costly to bootstrap/do a permutation test on anything which is also non-...
0
votes
0answers
29 views

Nonlinear Wald Test

Say we have a bivariate distribution $~~~~~f(x,y|\theta=\{a,b,t,c\}).$ An event of interest is when ab=c, that is, $~~~~~H_0: g(\theta)=c-ab=0$ versus $H_1: g(\theta)=c-ab \neq 0.$ I attempt to ...
0
votes
0answers
31 views

Power Calculations for 3 arm clustered randomised trial

I need to do power calculations for a 3 arm trial (to determine minimum detectable effect size, given the alpha, power, sample size etc). I found the R package Three.Arm.Trials, however I was ...
0
votes
0answers
38 views

Why does G*power calculate the noncentrality parameter in this way?(different from formula)

I'm calculating the power of some F-family tests and I found the key to the power was the noncentrality parameter (ncp). So I'm trying to calculate the ncp $\lambda$. However, I got different results ...
1
vote
0answers
36 views

Should a statistical test be performed if power is very low?

I performed some partitioning of data set (genes) into two lists, resulting in two enrichment lists. I wanted to compare them with Fisher's test, but from before I know that similarly looking results ...
1
vote
1answer
38 views

Statistically significant definition of power

After a not-quick literature research I noticed that there are two types od definition for what power is: One is: it's the probability of not making type II error' (A) The other is: it is the ...
0
votes
0answers
13 views

GLM - sample size estimation for given power and effect size and data set

I am working with R's car package highway1 and calculating a general linear model with rate as the dependant var (metric) and acpt as the independant var (metric). Additionally i used htype (...
2
votes
1answer
65 views

Binary Logit: A statistically insignificant predictor become significant after exclude some observation

I am currently trying to explore the determinants of management turnover using binary logit. In robustness check, I have added more control variable (Industry) to see whether the same results hold. ...
1
vote
1answer
43 views

Power & Effect Size for Multiple Regression - HELP

2 IV's: Pregnant/Non Pregnant & Physically Active/Not Physically Active 1 DV: Cognitive Functioning (Inhibition - SSRT scores) AIM: Investigate whether there is a difference in physical activity ...
5
votes
2answers
414 views

Power calculation for two-sample Welch's t test

The R function power.t.test does power calculations (outputs power, sample size, effect size, or whichever parameter you leave out) for t-tests, but only has a ...
1
vote
0answers
10 views

Implications of adding extra “control” conditions to an ANOVA?

I'm planning a 2x3 ANOVA, all between subjects. I'm expecting a weak main effect for A...a main effect for B where one condition is different than the other two...and then an interaction effect where ...
1
vote
1answer
54 views

Using the $Z$-test instead of the binomial test for small $n$ - effect on power and Type I error

Setup Let $M \sim \text{Bin}(p,n)$. Let us then state the following two hypotheses: $$\begin{align} H_0: p = 0.5 \text{ vs } H_1: p > 0.5 \end{align}$$ To test these hypotheses, I could use the ...
0
votes
0answers
14 views

Using power and significance to calculate sample size in a one-tailed test?

I am trying to determine the sample size needed to demonstrate that the lower boundary of a 95% CI is not lower than the population mean of another, with a power of .9. Right now the formula I'm using ...
3
votes
2answers
126 views

Importance sampling estimation of power function

Problem Suppose we are given $\text{Poisson}(\theta)$ model, and the null hypothesis is as follows: $$ H_0 : \theta = 0.1 \ \ \text{vs}. \ \ H_1 : \theta < 0.1 $$ Suppose we take sample of $n=...
1
vote
1answer
64 views

Power of a Poisson hypothesis test

A textbook of mine has online worked solutions, and the answer in the back of the book does not match the answer in the online solutions - they're not even close. My book says that the power of a ...
1
vote
0answers
140 views

simr power simulation testing interaction term for 2 factors

I have a model with two factors: a (4 levels) and b (3 levels). Each participant receives two problems (id as a random effect). I want to estimate sample size based on pilot data to give me ...
1
vote
1answer
25 views

Need help determining sample size for an AB test with two metrics

I need to run an experiment with customer support agents who are rated on two metrics by the customer: average issue resolution rate (proportion of issues resolved) and average star rating (1-5). ...
0
votes
1answer
200 views

Explanation of Advantages of Balanced Designs with Statistical Theory

I wanted to know what the difference between balanced and unbalanced design. Firstly, here are two advantages of balanced designs: ...
1
vote
1answer
26 views

Does a more powerful test mechanically lead to more type III errors then?

"Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. Say that a treatment increases some ...
0
votes
0answers
22 views

Power of Tests for Proportions: One Proportion vs. Two Proportions

I am trying to calculate the required sample size for a trial, where my outcome is proportions (for each subject I am measuring if the subject had an event or not). I tried two ways and got different ...
1
vote
2answers
50 views

Significance of effect size less then estimated by power

Suppose I estimate a sample size needed for a hypothesis test of proportion difference, using a binomial distributional assumption, with power 80% and effect size 25%. Then I perform an A/B ...
1
vote
1answer
58 views

Is Fisher's exact test the most powerful test for 2x2 contingency table?

I don't find the powerfulness of Fisher's exact test on Wikipedia. Are there any results about the power of FET vs other methods on 2x2 contingency tables?