# Questions tagged [power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

562 questions
Filter by
Sorted by
Tagged with
16 views

### Most powerful test of size zero for $\theta$ given random sample from $U(0, \theta)$

I found a couple of questions on this site (Most powerful test of simple vs. simple in $\mathrm{Unif}[0, \theta]$ and UMP for $U(0,\theta)$ (simple x simple hypothesis)) that are similar to my problem,...
6 views

### Simr: Post hoc power 0% and non signfiicant but effect size 0.3

I set up a glmer model via the package lme4. Using a likelihood ratio tests I compared several models. Two main effects were ...
55 views

### Interpretation of power calculations

We got the following comment from an associate editor who asked for ex-post power calculations. We used the treatment difference and sample size from our study and reported the power of detecting the ...
8 views

### Power calculation for a one-stage, single arm trial with continuous primary outcome

An early-stage single-arm clinical trial is being planned to evaluate the efficacy and safety of treatment for a bacterial infection. The primary outcome is binary - proportion with >0.5 log10 ...
17 views

### power analysis for correlations

Does anyone know of a good method to run a power analysis for correlations whilst taking into consideration the number of trials of the task? I have used the package pwr in R to look at power but it ...
7 views

### Calculate power for a cumulative link model, a categorial logistic regresssion in this case with G*Power

I want to calculate the power of a logit model. In R I used the clm function. I know, it is only possible to calculate an approximation. I want to use G*Power for the calculation. But I'm asked to ...
9 views

### How do you calculate cohen's L so that you can plug it into a power calculation for multiple linear regression?

I am reviewing a book called Statistical Methods for Healthcare Research so I can learn how to do a power calculation for my sample size for multiple linear regression. This book says that cohen ...
28 views

### What conclusions can I draw if my sample size is too small

I'm working on a paper involving multiple comparisons that have very small sample sizes (and no chance of increasing them). When I calculate p-values, they usually are not less than the usual .05-...
57 views

### Optimal unbalanced design for A/B test

I want to find the right way to distribute variants (control/test) in A/B-test with the constraint of no more 10 percent of population in test group. The test is t-test for continuous metric. I can ...
45 views

### Intuitive understanding of size and power of a test

Let $\mathcal{H}_0:\theta\in\Theta_0$ and $\mathcal{H}_1:\theta\in\Theta_1$ be the null and alternative hypotheses. The power of a test $\delta$ is defined as: \pi_{\delta}:\Theta\to[0,1] \ s.t.\ \...
7 views

### Fire an alert when number of sign up in an app drops. How to find the best condition to maximize accuracy?

I am writing alerts to monitor the sign up conversion rate for an app. Sign up conversion rate here means the percent of users that open up the app, who end up making an account. Usually, this is ...
28 views

### Minimum number of specimens for a medical experiment

Could someone validate the following - for the minimum number of specimens required. A pathogen has 8 different forms. Laboratory specimens (small organisms) are infected with the pathogen. For ...
19 views

### Calculating sample size required to test (power?) of tool

I have a tool that is designed to help people solve a puzzle. I want to see how good it is at helping people to do this. I thought I would start with taking a group of people and getting them to test ...
9 views

### Deriving level of significance from the power of the test [duplicate]

There are situations where researchers would like to specify the power of the test as opposed to specifying the level of significance. One such situation I have encountered involved testing the ...
22 views

### Relating $\alpha$, sample size ($n$), and power

I've seen in R-programming a pwr package (full disclosure, I don't know how to code or program). I was wondering if there is a formula and if possible a link to its derivation that lets you ...
29 views

### Finding $\beta$ as a function of $\alpha$ and $\mu$ in hypothesis testing

Question: Let $X_1,X_2,X_3,X_4 \stackrel{iid}\sim \mathrm N(\mu,1), \mu$ unknown. We want to decide between $H_0:\mu = 2$ and $H_1: \mu \ne 2$. If we require significance level $\alpha$, find $\beta$...
18 views

I am using G*power to do power calculations for univariate logistic regresion (in SPSS using GENLIN) and the results are baffling me despite a fair bit of reading ...
10 views

### How to calculate sample size of a study with difference in median as primary outcomes

A quick question on sample size calculation. How to calculate the sample size for a study in which the primary outcome is a difference in the median time to improvement (e.g. 10 median days in group A ...
9 views

### Power of Anderson-Darling with known vs unknown parameters

I recently ran some AD tests for normality on some data. I ran AD in two modes: (i) With specified (mean, sd) of the normal (ii) With unspecified (mean,sd) These correspond to cases 0 and 3, ...
46 views

### How to interpret underpowered significant study with high Positive Predictive Value

I've made a study about working memory and student grades. In the study, I wanted to see if there was a significant difference in working memory scores between students with higher grades compared to ...
17 views

### Significant p-value and anecdotical BF

I'm conducting frequentist as well as Bayesian analyses. However, I have difficulties to interpret data. Indeed, results showed significant p-value, but anecdotal BF10 (< 3). Likewise, there are ...
17 views

### Most Powerful Test for X~Poi($\lambda$)

Suppose $X \sim$ Pois($\mu$). a) Derive the most powerful test of $H_0 = \mu = \mu_0$ versus $H_a = \mu = \mu_1$ ($\mu_1>\mu_0$) based on an observed value of X. b) Rework a) based on a random ...
42 views

### Power analysis for cluster-robust regression

I am planning to run a regression for the model of the form: $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_1x_2 + \epsilon$ To determine the necessary sample size, I would usually use R's ...
41 views

### Power simulation on glmer.nb gave strange results

I would like to ask for solution or advice on strange result that glmer.nb from lme4 generated when simulating using simR package. I’m working on longitudinal gut microbiome abundance data (23 ...
16 views

### Significance for low sample sizes

The question is - formally speaking, what should the minimum sample size be in order to trust statistical metrics like p-value and power? Let's say we have population A and B each having values ...
45 views

### Power of a test in the case of Bernoulli sample

I'm solving the following exercise: We consider a sample $X_{1}, \cdots, X_{n}$ of size $n$ of the Bernoulli distribution with parameter $\theta,$ where $0<\theta<1$ Prove that this family of ...
74 views

### Wilcoxon Rank Sum Test vs $t$-test Power Simulation

It is well known that the Wilcoxon rank sum test is more powerful for detecting shifts in location when the data is non-normal. However, I am conducting a brief simulation study and my results ...
17 views

### Simple random sample size for large data question and appropriate strategy

I am working with a large data set from my company. I am trying to determine if using another sample would provide a better coverage compared to what I am using now. To do so, I am going to take a ...
11 views

### How can I assess the number of cases needed to distinguish whether responses are from one of several probability distributions?

In my experiment, participants choose one of three options on every trial (the trials are 3x3 versions of the well known Prisoner's Dilemma game). What decision rule they use determines the ...
43 views

### ROC Curve AUC for Hypothesis Testing Sensitivity (Power) vs Specificity ($1-\alpha$)

Let's set up a hypothesis test of $H_0: \theta=\theta_0$ versus $H_1: \theta\ne\theta_0$, and let's say that I have two techniques to assess this (say equal-variance t-test versus unequal-variance ...
27 views

### power analysis for difference of means — without examining distribution of difference of means

I am interested in calculating the power of studies to detect differences of means with a z- or a t-test -- and in doing so without directly examining the sampling distribution of the difference of ...
28 views

### AB Test Cross Contamination

I was talking with consultancy today about running geo experiments. The idea is to randomise on a zip code level instead of a user level for exposing customers to ads online. The issue is, there will ...
17 views

### Determine necessary sample needed to assess true positives

I am trying to determine what is the sample I need in order to determine if a new method is as effective as an old one. So for example, in three weeks samples were drawn of 234, 229, and 163. These ...
25 views

### Power analysis of binary predictor in cohort

I'm trying to run a power analysis in Stata, using chi2 (even though I am using logistic regression, I just can't find an option for it and I'm new to power analysis) and it seems I have to choose how ...
327 views

### How to interpret sensitivity power analyses?

Journals and reviewers increasingly ask authors to systematically report sensitivity power analyses. I know that a sensitivity power analysis allows you to determine the minimum effect size that the ...
20 views

### Power of smooth Neyman Test

I would like to create code in R for POWER of Neyman's Smooth Test (with k components), which is generate from pdf: $C_1(u,0.4) = 1+0.4cos(\pi u), u\in (0,1)$. I generate $n$ observation and repeat in ...
19 views

### What are other uniformly most powerful tests besides the likelihood ratio test?

What are other uniformly most powerful tests besides the likelihood ratio test? Are there infinitely many known ones? The Wald and score test are asymptotically as powerful as the likelihood ratio ...
2k views

### What does it mean for a statistical test to have a power of 0.8?

The power is a continuous function of the parameter. If a statistical test has a power of 0.8, does that mean the power function is 0.8 at all parameters except the null hypothesis? Or does it mean ...
23 views

### In survival analysis, how to calculate power from hazard ratio and number of events given alpha?

If I know number of events and hazard ratio, with given alpha, how do I calculate the power?
62 views

### Is it possible to generate power curves for score-tests, wald-test, LR test?

A power curve is something in the example Do score-tests, wald-test, LR test have power curves? What do they look like?
5 views

### Optimal Power Value

I am doing a hypothesis test, with h0: p=0.5, with H1:p>0.5. If I want to test different values of p that are greater than 0.5, with a fixed significance level, say 0.05, how does the choice of p ...
270 views

### What does good power curve or bad power curve for some hypothesis tests look like?

What does the most ideal power curve look like? What does the worst power curve look like? Does there exist good power curves or bad power curves? Obviously there exists more powerful and less ...
11 views

### rule of thumb for generalizing observations to population

I recently had a question from a co-worker, “How big a sample would you need to feel comfortable communicating figures?” My answer was two part: 1) if we’re trying to show something made a ...
13 views

### Can pooled standard deviation be used to determine sample size?

I would like to use Minitab statistical software to determine a minimum sample size for precision (repeatability) testing. From casual observation, we know our allowable error is much wider than the ...
16 views

### Power function on a calculator (=normalcdf)

I am preparing myself for a statistics exam and got stuck on a question. Namely, when asked to calculate the power 1-beta of a statistical test I see that in solutions to some exercises they use 1-...
23 views

### How to calculate sample size in a dataset with multiple tests

i need some help. I just cannot figure it out on my own. I got told to find out if we already have enough participants (birds) for this study, so do a power analysis to calculate sample size. ...
29 views

### Power calculation & study design for study with rare safety outcome

An RCT is planned to compare a new/repurposed drug to placebo on the basis of mechanistic findings. Limited safety data exist. The primary efficacy outcome requires the recruitment of ~200 ...
20 views

### required sample size for establishing equivalence of a gradient boosting model on different population

I have a trained Gradient boosting Trees (regression) model with a given R2 metric (obtained via cross validation) Now I want to verify that the same model is valid for a very different population. Is ...