Questions tagged [power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

Filter by
Sorted by
Tagged with
2
votes
1answer
41 views

Power calculation for two-sample Welch's t test

The R function power.t.test does power calculations (outputs power, sample size, effect size, or whichever parameter you leave out) for t-tests, but only has a ...
1
vote
0answers
10 views

Implications of adding extra “control” conditions to an ANOVA?

I'm planning a 2x3 ANOVA, all between subjects. I'm expecting a weak main effect for A...a main effect for B where one condition is different than the other two...and then an interaction effect where ...
1
vote
1answer
45 views

Using the $Z$-test instead of the binomial test for small $n$ - effect on power and Type I error

Setup Let $M \sim \text{Bin}(p,n)$. Let us then state the following two hypotheses: $$\begin{align} H_0: p = 0.5 \text{ vs } H_1: p > 0.5 \end{align}$$ To test these hypotheses, I could use the ...
0
votes
0answers
12 views

Using power and significance to calculate sample size in a one-tailed test?

I am trying to determine the sample size needed to demonstrate that the lower boundary of a 95% CI is not lower than the population mean of another, with a power of .9. Right now the formula I'm using ...
3
votes
2answers
118 views

Importance sampling estimation of power function

Problem Suppose we are given $\text{Poisson}(\theta)$ model, and the null hypothesis is as follows: $$ H_0 : \theta = 0.1 \ \ \text{vs}. \ \ H_1 : \theta < 0.1 $$ Suppose we take sample of $n=...
1
vote
0answers
62 views

Calculate power for my test [closed]

I need some help with the following problem: Let's consider a density function $f_\theta(x):=\theta x^{\theta -1}$, with $x \in [0, 1]$, the null hypothesis $H_0: \theta=2$ and the alternative ...
1
vote
1answer
51 views

Power of a Poisson hypothesis test

A textbook of mine has online worked solutions, and the answer in the back of the book does not match the answer in the online solutions - they're not even close. My book says that the power of a ...
1
vote
0answers
31 views

simr power simulation testing interaction term for 2 factors

I have a model with two factors: a (4 levels) and b (3 levels). Each participant receives two problems (id as a random effect). I want to estimate sample size based on pilot data to give me ...
1
vote
1answer
20 views

Need help determining sample size for an AB test with two metrics

I need to run an experiment with customer support agents who are rated on two metrics by the customer: average issue resolution rate (proportion of issues resolved) and average star rating (1-5). ...
0
votes
1answer
13 views

Explanation of Advantages of Balanced Designs with Statistical Theory

I wanted to know what the difference between balanced and unbalanced design. Firstly, here are two advantages of balanced designs: ...
1
vote
1answer
25 views

Does a more powerful test mechanically lead to more type III errors then?

"Type III error occurs when you correctly conclude that the two groups are statistically different, but you are wrong about the direction of the difference. Say that a treatment increases some ...
0
votes
0answers
14 views

Power of Tests for Proportions: One Proportion vs. Two Proportions

I am trying to calculate the required sample size for a trial, where my outcome is proportions (for each subject I am measuring if the subject had an event or not). I tried two ways and got different ...
1
vote
2answers
35 views

Significance of effect size less then estimated by power

Suppose I estimate a sample size needed for a hypothesis test of proportion difference, using a binomial distributional assumption, with power 80% and effect size 25%. Then I perform an A/B ...
1
vote
1answer
32 views

Is Fisher's exact test the most powerful test for 2x2 contingency table?

I don't find the powerfulness of Fisher's exact test on Wikipedia. Are there any results about the power of FET vs other methods on 2x2 contingency tables?
0
votes
0answers
14 views

A question about power for a cohort study

In short, we are planing a cohort study, were we want to investigate the occurrence and the importance of a given event on response of a treatment. We expect the event to occur in about 2% of all the ...
0
votes
0answers
26 views

Plot power of a hypothesis test with $\alpha$

Let's take a simple one-sided hypothesis test (say the two sample t-test). The confidence level we want (probability of rejecting the null when it is true) is defined as $\alpha$ - usually set to 5%. ...
1
vote
1answer
28 views

sample size calculation in 3-arm survival analysis

In a 3-arms clinical trial, with time to event data, either A is superior to C or B is superior to C it will be considered significant. No need to compare A and B. I know how to calculate the sample ...
0
votes
0answers
13 views

How does variation in the effect size affects power?

Effect size The effect size affect the power of a statistical test. We typically summarizes the magnitude of the effect of a variable as a single number (which we call the effect size). To my ...
1
vote
0answers
22 views

How to calculate a sample size for a longitudinal study with repeated data?

Did you know a program or logiciel that allows to calculate the sample size for a longitudinal study with repeated data ? And if my subjects are nested in some schools, how I calculate the sample ...
0
votes
1answer
20 views

Compute an approximate power when lambda = 10

I wanna reconfirm, is this the right way to answer this question? I'm pretty confused. And if not, how?: Answer: Thank you so much.
0
votes
0answers
11 views

Statistical Power alternatives for post-hoc analysis

I am reading about Power and how it is misleading/uninformative to calculate a post-hoc "observed power" of a study. I would still like, after reading a study, to be able to judge its ability to have ...
2
votes
1answer
29 views

Why is my sample size so big after doing pwr.t.test() in R?

For a research I want to do a t-test between two populations. Since I have a fairly small sample size I want to calculate the necessary sample-size for getting a good power (0.8 is supposed to be very ...
0
votes
0answers
14 views

Determine PPV from type I and type II error rates

Consider these two scenarios: In both of these scenarios, flipping heads 60 times (the X-axis should read # of heads, not # of coin flips) would be reported as a p of ~0.05 if we are using the fair ...
0
votes
0answers
9 views

Comparing two methods of measurement

In relation to testing a new method of measuring a variable, I am interested in knowing how accurate the new method is. With one method (M1), we measure the "ground-truth": A variable with normal ...
1
vote
1answer
47 views

Power analysis for binomial regression (success/failure)

I've collect 161 people for a study which the original power analysis was based on correlations, but now I've realised binomial regression would be better. I can find a lot of articles on comparing ...
0
votes
1answer
46 views

Required sample size to reach adequate power of a test

We got two types of treatment (type 1 and type 2). In our opinion, type 2 treatment has a higher performance and we would like to show this with research. We assume that type 1 treatment has a maximum ...
2
votes
0answers
48 views

An alternative definition of the $p$-value

As we know, $p$-values are uniformly distributed under $H_0$. This urges me to ask if this constitutes a valid (re-)definition of the $p$-value. $p$-value - A statistic with a uniform distribution ...
0
votes
0answers
6 views

How to combine Benjamini-Hochberg with statistical power?

Suppose I am investigating a question which involves many statistical T-tests. The normal Benjamini-Hochberg procedure tells me how to control the false discovery rate. However, suppose that some or ...
2
votes
2answers
154 views

Can bootstrapping improve the test power, when the sample size is small? [closed]

When the sample is small and study is under-powered, can bootstrapping be used to generate data and increase the sample size (and improve the test power)? Is there any way that bootstrapping can be ...
0
votes
0answers
22 views

Shapiro–Wilk test for non-normal distributions

The Shapiro–Wilk test makes use of the correlation between the data (a.k.a. sample quantile) $x$ and its theoretical quantile $m$. As far as I'm concerned, it's a normality test simply because the ...
0
votes
1answer
25 views

Computing the power of Welch's t-test

I constructed a Welch's t-test in order to examine the difference between the means of two samples (check whether or not one is larger than the other). $H_{0}: \mu_0 = \mu_1 ~\textrm{vs}~ H_{1}: \...
0
votes
0answers
15 views

How do I calculate the statistical reliability of a random sampling of features?

I have a dataset of 6 million features from which I've trained a binary classification model. It identified 32 features that can accurately predict 98% of samples (ie if I train a model with only ...
0
votes
0answers
12 views

How to calculate statistical power in a clustering problem?

I'm considering following question. I implement a algorithm to cluster my data and the result will assign each cluster to a small cluster or big cluster according to its size. After that, I will get ...
0
votes
0answers
45 views

Power analysis for ordinal logistic regression (multiple independent variables)

I have a question with regard to a simulation based power analysis for ordinal logistic regression suggested here: https://stats.stackexchange.com/a/22410/231675 (by Greg Snow) The suggested power ...
1
vote
2answers
86 views

proof for the hypothesis testing $\alpha-\beta$ tradeoff

All the statistic's courses state that there is a tradeoff between the significance level ($\alpha$) and the probability of the Type II error ($\beta$). Specifically, for a fixed sample size, one ...
2
votes
1answer
67 views

Significance Tests - Why Would a Low Sample Size Cause Problems?

Okay, on my AP Statistics test on the chapter on significance tests, the following free response question appeared: When the manufacturing process is working properly, NeverREady batteries have ...
0
votes
0answers
82 views

Non-Parametric Effect Sizes and Power

As my data was extremely skewed, I have ran 2 non-parametric tests: a Mann-Whitney U and a Wilcoxon Signed Rank test. I have read that using Cohen's D as an effect size for such tests is not ...
0
votes
1answer
36 views

How exactly do I calculate the power in my SEM?

I am trying to calculate the power in my SEM analysis post-hoc. How exactly should I do this? What is the power for the R-squared result of IT-T2 and IT-T3? Background info: Sample size is 255. IT ...
1
vote
0answers
44 views

UMP test for $H_0:p=0.5$ vs $H_1:p\neq0.5$?

Let $X_1,\dots, X_n$ Bernoulli trials. I know that the UMP tests for $$H_0:p=0.5 \quad\text{vs}\quad H_1:p>0.5$$ and $$H_0:p=0.5 \quad\text{vs}\quad H_1:p<0.5$$ can be obtained with the Neyman ...
0
votes
0answers
45 views

Possible to extend Gelman & Carlin's “Beyond Power Calculations” when significance not obtained?

I have recently been spending time trying to understand the content and extent/limitations of this paper: "Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors" by Gelman ...
0
votes
0answers
11 views

Formula for Power of Upper Tail Sign Test

Given Consider the upper tail $H_0: \theta \leq 0 \,\,\, \text{vs} \,\,\, H_A: \theta > 0$ sign test with the test statistic $B = \sum_{i=1}^n{\psi_i}$ where $\psi_i = \mathbb{I}(Z_i > \theta)$...
1
vote
0answers
25 views

Sample size formulae

In order to calculate sample size for hypothesis testing, we consider level of significance and power. For level of significance, two sided values ($Z_{\alpha/2}$) are taken into account, but for ...
2
votes
1answer
27 views

What causes low power on hypothesis tests?

I've been doing hypothesis tests to compare two proportions in a product. Although the sample size is supposed to be more than enough based on the population and $\alpha$ (type I error), the power of ...
5
votes
1answer
182 views

AB test sample size calculation by hand

Evan Miller has created a well-known online AB test sample size calculator. For the sake of being able to program and modify this formula, I would like to know how to calculate sample size Evan Miller-...
1
vote
0answers
33 views

Power calculations for moderator analyses in RCT

I am running a randomised controlled trial of a savings intervention in India whereby randomisation is at the individual level (not clusters). In my power calculations, I also want to factor in a ...
1
vote
1answer
35 views

Power analysis for t-test instead of ANOVA

Let's say I have an experiment with a 2x2 repeated-measures design. Let's call the four cells A1B1, A1B2, A2B1, A2B2, for the factors A and B, each with two conditions, labeled 1 and 2. I want to ...
3
votes
2answers
69 views

Can one make statistical inferences from 6 patients?

Is it conceptually correct to make statistical inferences from a very small number of patients (even when using exact methods like Fisher's test, etc)? What do I have to look out for: power or p-value?...
3
votes
2answers
1k views

Why this simple mixed model fail to converge?

I have data on which I would like to compare the mean of the value variable between day 28 and day 83. Because the experience involve pseudo-replication (...
0
votes
0answers
25 views

Sensitivity and Specificity of gaussian and negative binomial glm family

I am dealing with count data that is over-dispersed and hence I consider using the negative binomial family (with glm()) instead of the gaussian family, as the link function. To find out the ...
2
votes
0answers
27 views

How does one perform a conditional power analysis?

Suppose a test statistic $Z$ will be compared to a standard normal distribution to evaluate whether a $p$-value achieves a statistically significant result at the $\alpha$-level. Based on an ...