Questions tagged [power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

Filter by
Sorted by
Tagged with
3
votes
1answer
16 views

Most powerful test of size zero for $\theta$ given random sample from $U(0, \theta)$

I found a couple of questions on this site (Most powerful test of simple vs. simple in $\mathrm{Unif}[0, \theta]$ and UMP for $U(0,\theta)$ (simple x simple hypothesis)) that are similar to my problem,...
0
votes
0answers
6 views

Simr: Post hoc power 0% and non signfiicant but effect size 0.3

I set up a glmer model via the package lme4. Using a likelihood ratio tests I compared several models. Two main effects were ...
1
vote
2answers
55 views

Interpretation of power calculations

We got the following comment from an associate editor who asked for ex-post power calculations. We used the treatment difference and sample size from our study and reported the power of detecting the ...
1
vote
0answers
8 views

Power calculation for a one-stage, single arm trial with continuous primary outcome

An early-stage single-arm clinical trial is being planned to evaluate the efficacy and safety of treatment for a bacterial infection. The primary outcome is binary - proportion with >0.5 log10 ...
0
votes
0answers
17 views

power analysis for correlations

Does anyone know of a good method to run a power analysis for correlations whilst taking into consideration the number of trials of the task? I have used the package pwr in R to look at power but it ...
0
votes
0answers
7 views

Calculate power for a cumulative link model, a categorial logistic regresssion in this case with G*Power

I want to calculate the power of a logit model. In R I used the clm function. I know, it is only possible to calculate an approximation. I want to use G*Power for the calculation. But I'm asked to ...
1
vote
0answers
9 views

How do you calculate cohen's L so that you can plug it into a power calculation for multiple linear regression?

I am reviewing a book called Statistical Methods for Healthcare Research so I can learn how to do a power calculation for my sample size for multiple linear regression. This book says that cohen ...
2
votes
0answers
28 views

What conclusions can I draw if my sample size is too small

I'm working on a paper involving multiple comparisons that have very small sample sizes (and no chance of increasing them). When I calculate p-values, they usually are not less than the usual .05-...
3
votes
2answers
57 views

Optimal unbalanced design for A/B test

I want to find the right way to distribute variants (control/test) in A/B-test with the constraint of no more 10 percent of population in test group. The test is t-test for continuous metric. I can ...
2
votes
1answer
45 views

Intuitive understanding of size and power of a test

Let $\mathcal{H}_0:\theta\in\Theta_0$ and $\mathcal{H}_1:\theta\in\Theta_1$ be the null and alternative hypotheses. The power of a test $\delta$ is defined as: $$\pi_{\delta}:\Theta\to[0,1] \ s.t.\ \...
0
votes
0answers
7 views

Fire an alert when number of sign up in an app drops. How to find the best condition to maximize accuracy?

I am writing alerts to monitor the sign up conversion rate for an app. Sign up conversion rate here means the percent of users that open up the app, who end up making an account. Usually, this is ...
2
votes
1answer
28 views

Minimum number of specimens for a medical experiment

Could someone validate the following - for the minimum number of specimens required. A pathogen has 8 different forms. Laboratory specimens (small organisms) are infected with the pathogen. For ...
0
votes
0answers
19 views

Calculating sample size required to test (power?) of tool

I have a tool that is designed to help people solve a puzzle. I want to see how good it is at helping people to do this. I thought I would start with taking a group of people and getting them to test ...
0
votes
0answers
9 views

Deriving level of significance from the power of the test [duplicate]

There are situations where researchers would like to specify the power of the test as opposed to specifying the level of significance. One such situation I have encountered involved testing the ...
0
votes
1answer
22 views

Relating $\alpha$, sample size ($n$), and power

I've seen in R-programming a pwr package (full disclosure, I don't know how to code or program). I was wondering if there is a formula and if possible a link to its derivation that lets you ...
0
votes
0answers
29 views

Finding $\beta$ as a function of $\alpha$ and $\mu$ in hypothesis testing

Question: Let $X_1,X_2,X_3,X_4 \stackrel{iid}\sim \mathrm N(\mu,1), \mu$ unknown. We want to decide between $H_0:\mu = 2 $ and $ H_1: \mu \ne 2$. If we require significance level $\alpha$, find $\beta$...
0
votes
0answers
18 views

Contradictory G*Power logistic regression

I am using G*power to do power calculations for univariate logistic regresion (in SPSS using GENLIN) and the results are baffling me despite a fair bit of reading ...
1
vote
0answers
10 views

How to calculate sample size of a study with difference in median as primary outcomes

A quick question on sample size calculation. How to calculate the sample size for a study in which the primary outcome is a difference in the median time to improvement (e.g. 10 median days in group A ...
0
votes
0answers
9 views

Power of Anderson-Darling with known vs unknown parameters

I recently ran some AD tests for normality on some data. I ran AD in two modes: (i) With specified (mean, sd) of the normal (ii) With unspecified (mean,sd) These correspond to cases 0 and 3, ...
0
votes
0answers
46 views

How to interpret underpowered significant study with high Positive Predictive Value

I've made a study about working memory and student grades. In the study, I wanted to see if there was a significant difference in working memory scores between students with higher grades compared to ...
0
votes
0answers
17 views

Significant p-value and anecdotical BF

I'm conducting frequentist as well as Bayesian analyses. However, I have difficulties to interpret data. Indeed, results showed significant p-value, but anecdotal BF10 (< 3). Likewise, there are ...
0
votes
0answers
17 views

Most Powerful Test for X~Poi($\lambda$)

Suppose $X \sim$ Pois($\mu$). a) Derive the most powerful test of $H_0 = \mu = \mu_0$ versus $H_a = \mu = \mu_1$ ($\mu_1>\mu_0$) based on an observed value of X. b) Rework a) based on a random ...
1
vote
1answer
42 views

Power analysis for cluster-robust regression

I am planning to run a regression for the model of the form: $y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \beta_3x_1x_2 + \epsilon$ To determine the necessary sample size, I would usually use R's ...
2
votes
1answer
41 views

Power simulation on glmer.nb gave strange results

I would like to ask for solution or advice on strange result that glmer.nb from lme4 generated when simulating using simR package. I’m working on longitudinal gut microbiome abundance data (23 ...
0
votes
0answers
16 views

Significance for low sample sizes

The question is - formally speaking, what should the minimum sample size be in order to trust statistical metrics like p-value and power? Let's say we have population A and B each having values ...
0
votes
0answers
45 views

Power of a test in the case of Bernoulli sample

I'm solving the following exercise: We consider a sample $X_{1}, \cdots, X_{n}$ of size $n$ of the Bernoulli distribution with parameter $\theta,$ where $0<\theta<1$ Prove that this family of ...
4
votes
1answer
74 views

Wilcoxon Rank Sum Test vs $t$-test Power Simulation

It is well known that the Wilcoxon rank sum test is more powerful for detecting shifts in location when the data is non-normal. However, I am conducting a brief simulation study and my results ...
0
votes
0answers
17 views

Simple random sample size for large data question and appropriate strategy

I am working with a large data set from my company. I am trying to determine if using another sample would provide a better coverage compared to what I am using now. To do so, I am going to take a ...
0
votes
0answers
11 views

How can I assess the number of cases needed to distinguish whether responses are from one of several probability distributions?

In my experiment, participants choose one of three options on every trial (the trials are 3x3 versions of the well known Prisoner's Dilemma game). What decision rule they use determines the ...
1
vote
1answer
43 views

ROC Curve AUC for Hypothesis Testing Sensitivity (Power) vs Specificity ($1-\alpha$)

Let's set up a hypothesis test of $H_0: \theta=\theta_0$ versus $H_1: \theta\ne\theta_0$, and let's say that I have two techniques to assess this (say equal-variance t-test versus unequal-variance ...
2
votes
0answers
27 views

power analysis for difference of means — without examining distribution of difference of means

I am interested in calculating the power of studies to detect differences of means with a z- or a t-test -- and in doing so without directly examining the sampling distribution of the difference of ...
0
votes
0answers
28 views

AB Test Cross Contamination

I was talking with consultancy today about running geo experiments. The idea is to randomise on a zip code level instead of a user level for exposing customers to ads online. The issue is, there will ...
0
votes
0answers
17 views

Determine necessary sample needed to assess true positives

I am trying to determine what is the sample I need in order to determine if a new method is as effective as an old one. So for example, in three weeks samples were drawn of 234, 229, and 163. These ...
1
vote
1answer
25 views

Power analysis of binary predictor in cohort

I'm trying to run a power analysis in Stata, using chi2 (even though I am using logistic regression, I just can't find an option for it and I'm new to power analysis) and it seems I have to choose how ...
1
vote
1answer
327 views

How to interpret sensitivity power analyses?

Journals and reviewers increasingly ask authors to systematically report sensitivity power analyses. I know that a sensitivity power analysis allows you to determine the minimum effect size that the ...
0
votes
0answers
20 views

Power of smooth Neyman Test

I would like to create code in R for POWER of Neyman's Smooth Test (with k components), which is generate from pdf: $C_1(u,0.4) = 1+0.4cos(\pi u), u\in (0,1)$. I generate $n$ observation and repeat in ...
1
vote
0answers
19 views

What are other uniformly most powerful tests besides the likelihood ratio test?

What are other uniformly most powerful tests besides the likelihood ratio test? Are there infinitely many known ones? The Wald and score test are asymptotically as powerful as the likelihood ratio ...
10
votes
2answers
2k views

What does it mean for a statistical test to have a power of 0.8?

The power is a continuous function of the parameter. If a statistical test has a power of 0.8, does that mean the power function is 0.8 at all parameters except the null hypothesis? Or does it mean ...
1
vote
1answer
23 views

In survival analysis, how to calculate power from hazard ratio and number of events given alpha?

If I know number of events and hazard ratio, with given alpha, how do I calculate the power?
3
votes
2answers
62 views

Is it possible to generate power curves for score-tests, wald-test, LR test?

A power curve is something in the example Do score-tests, wald-test, LR test have power curves? What do they look like?
0
votes
0answers
5 views

Optimal Power Value

I am doing a hypothesis test, with h0: p=0.5, with H1:p>0.5. If I want to test different values of p that are greater than 0.5, with a fixed significance level, say 0.05, how does the choice of p ...
3
votes
1answer
270 views

What does good power curve or bad power curve for some hypothesis tests look like?

What does the most ideal power curve look like? What does the worst power curve look like? Does there exist good power curves or bad power curves? Obviously there exists more powerful and less ...
0
votes
0answers
11 views

rule of thumb for generalizing observations to population

I recently had a question from a co-worker, “How big a sample would you need to feel comfortable communicating figures?” My answer was two part: 1) if we’re trying to show something made a ...
1
vote
0answers
13 views

Can pooled standard deviation be used to determine sample size?

I would like to use Minitab statistical software to determine a minimum sample size for precision (repeatability) testing. From casual observation, we know our allowable error is much wider than the ...
0
votes
0answers
16 views

Power function on a calculator (=normalcdf)

I am preparing myself for a statistics exam and got stuck on a question. Namely, when asked to calculate the power 1-beta of a statistical test I see that in solutions to some exercises they use 1-...
0
votes
0answers
23 views

How to calculate sample size in a dataset with multiple tests

i need some help. I just cannot figure it out on my own. I got told to find out if we already have enough participants (birds) for this study, so do a power analysis to calculate sample size. ...
0
votes
1answer
29 views

Power calculation & study design for study with rare safety outcome

An RCT is planned to compare a new/repurposed drug to placebo on the basis of mechanistic findings. Limited safety data exist. The primary efficacy outcome requires the recruitment of ~200 ...
0
votes
0answers
20 views

required sample size for establishing equivalence of a gradient boosting model on different population

I have a trained Gradient boosting Trees (regression) model with a given R2 metric (obtained via cross validation) Now I want to verify that the same model is valid for a very different population. Is ...
0
votes
1answer
24 views

How can one define the power of a randomized clinical trial?

The statistical power of a study is a measure of the ability of a randomized clinical trial (RCT) to detect a difference statistically significant of the treatment effect in the treatment arms. How ...
1
vote
0answers
20 views

Does the statistical power for testing regression coefficients depend on the number of model parameters?

Given a generalized linear model (GLM), does the number of regression coefficients affect the power to detect a significant effect size for an individual regression coefficient i.e. as more variables ...

1
2 3 4 5
12