Questions tagged [statistical-power]

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

Validity of R pwr package for extremely small sample sizes (<10)

Is the pwr package implemented with the small sample approximations (e.g. degrees of freedom in 1-sample t-test) described in Cohen 1988 or not? I can’t seem to ...
W_vH's user avatar
  • 96
0 votes
0 answers
32 views

Number of levels for ordinal DV, variance, and statistical significance

I am interested in how closely a person places themselves to a certain situation in the responses they give. The responses are coded, and I create a score, with the lowest value indicating the person ...
grey's user avatar
  • 645
0 votes
0 answers
47 views

Post hoc power for multiple logistic regression

I have data frame named "finalend", I conducted multiple logistic regression model named "model" with all predictors as categorical or binomial variables and "TestAnxiety"...
NEA's user avatar
  • 33
1 vote
1 answer
50 views

What would "small", "medium," and "large" effect sizes be for mixed effects model in simr?

I'm trying to perform an a-priori power analysis for a mixed effects model using the simr package, but I'm not sure which effect size to choose. When performing a power analysis for ANOVA I've used ...
mels's user avatar
  • 37
5 votes
2 answers
533 views

How does reducing my alpha-threshold to 0.005 affect my required sample size?

Among statisticians there is an ongoing debate whether the long-standing "standard" threshold for declaring the outcome of a statistical test as "significant" (a.k.a. alpha-level) ...
PuzzledBiologist's user avatar
0 votes
0 answers
16 views

Power analysis compatible with package glmmTMB

I am trying to conduct a power analysis on the following model: NativeAntModel <- glmmTMB(NativeAnts~ Treatment + Month + (1|Site), data = NativeAntData, family = t_family) Background on the data: ...
kira's user avatar
  • 1
0 votes
0 answers
11 views

Sample size requirements for evaluating a many-class classifier

I am working on defining the requirements for a test/evaluation dataset for a many-class classifier (n_classes ~ 1,000), and am working to address concerns about ensuring the statistical power of the ...
jf2qm's user avatar
  • 1
6 votes
1 answer
78 views

Power of two-sample z-test

In a pilot study with two groups, the control distribution has the Mean1 = 90, SD1 = 5 and the treatment distribution has Mean2 = 85, SD2 = 5. The null hypothesis of the test is that the sample ...
Christian's user avatar
  • 183
0 votes
0 answers
25 views

What does 'Exchangeability' mean in multiple testing?

In A practical guide to methods controlling false discoveries in computational biology, the author mentioned 'Exchangeability': According to Wiki, the definition of 'Exchangeability' is : ...
doraemon's user avatar
  • 208
0 votes
0 answers
41 views

Sample size calculation for logistic regression [duplicate]

I need to perform an a priori power analysis for a logistic regression. My dependent variable is binary and I have two independent variables (between subjects), one variable with 3 levels (group) and ...
Tes's user avatar
  • 1
0 votes
1 answer
50 views

How to calculate sample size from hazard ratio?

The marker gene XYX1 being high in breast cancer is associated with poor prognosis with higher hazard ratio : 3.5. I want to calculate sample size with 1 - β (power) = 80%, one sided, type I error? ...
svp's user avatar
  • 103
1 vote
2 answers
186 views

Determination of sample size vs. repeated measurements in experimental design

NOTE: Revised question following suggestions from others. Thank you for taking the time to help me refine this question to make it answerable! Background I want to compare the performance of two ...
CBRF23's user avatar
  • 61
1 vote
0 answers
43 views

Deriving Sample Size for two samples t test

I am trying to figure out what sample size recommendations to make in this case when the samples have different sizes. Since their sizes are different the variance of each will be different. Suppose ...
Harry Lofi's user avatar
1 vote
1 answer
42 views

Should we favor family-wise error rate control or statistical power?

To counteract the multiple comparison problem, people generally use procedures that ensure strong level alpha FWER control (e.g., Bonferroni, Holm). However, these procedures necessarilly produce a ...
user1363251's user avatar
0 votes
0 answers
10 views

Determining power under the assumption that Z_0(test statistic) has the variance of 1

My apology if there already are similar questions. On statistical hypothesis testing; When you design the test for the estimate of binomially-distributed population's mean and are determining the ...
Bipod's user avatar
  • 1
2 votes
1 answer
47 views

Calculate sample size in an experiment with interactions

I need to conduct a power analysis, but I'm not sure if I'm doing it correctly. Initially, I have four units, and each unit will be divided into two halves. In Units 1 and 2, stimulation (S) will be ...
aa3ls's user avatar
  • 23
2 votes
4 answers
77 views

How can power and effect size help me interpret my A/B test results?

To give an idea of my use case, I'm performing an A/B test using a two-sample t-test to compare the mean between a control and a treatment sample. I set $\alpha=0.05$. I set my effect size $d=0.01$ ...
KiwiKiwi's user avatar
4 votes
2 answers
185 views

What is the optimal method for distinguishing lack of power from non-significance in linear mixed models?

I have a linear mixed model and am interested in the significance of the fixed effects. I run run 2 identical models, except the second eliminates data points that are outliers (i.e., a sensitivity ...
SilvaC's user avatar
  • 472
1 vote
0 answers
38 views

Total N patients for precision around a rate, with varying FU times

To design a retrospective longitudinal study (outcome: certain event after drug exposure), we know that the annualized rate of that event is 0.06%, aka 0.0006/person-year (possibly underestimated as ...
Alex Ortega's user avatar
2 votes
0 answers
55 views

Power sizing when sampling from a group with known subgroups

I need to do a power analysis of a group that is composed of three subgroups. The measurements to sample are difference measurements between two dogs of the same breed rated side by side. Dogs are ...
Estimate the estimators's user avatar
2 votes
0 answers
47 views

TOST procedure + power analysis: how to interpret t-value inside bounds

I conducted a TOST procedure according to the guidelines by Lakens [1] and received the results from the table below. For "discontinuity", "fragmentation" and "...
Leon Schreiber's user avatar
1 vote
0 answers
35 views

How do I calculate the statistical power of my survey results? It is an exploratory survey with no measurements

I need to prove that my survey results or sample size is statistically significant. I did a 2 sample T test in R, where my supervisor said that "what is the statistical power of your results is ...
Maha's user avatar
  • 11
3 votes
1 answer
86 views

How to calculate sample size? How to affect sample size?

We are trying to figure out sample size to our research. We are comparing to groups. Group 1 will get treatment and group 2 don't get treatment. From previous studies we know that there was recurrence ...
user405350's user avatar
2 votes
1 answer
74 views

Power analysis for a quasi-experimental factorial design 3(manipulated) x 1 (measured) ANCOVA

I'm experiencing an hard time looking for the proper analysis to use to compute the power for an experiment i'd like to implement, so here is my question for you. I'd like to run a quasi-experimental ...
GT87's user avatar
  • 143
0 votes
0 answers
19 views

Dealing with imbalanced datasets for a case-control study [duplicate]

I'm working with a dataset where the number of samples in my positive class is significantly lower than the negative class (e.g., 30 positive samples and 160 negative samples). I am planning to ...
Ed9012's user avatar
  • 181
3 votes
0 answers
15 views

Power calculation with multiple techniques and multiple observations per subject

Say I have C algorithms/techniques and N subjects. For each algorithm/technique I make some (variable) number of measurements ...
sudo make install's user avatar
1 vote
0 answers
37 views

Repetitions on the same combination of factors in mixed ANOVA and its power analysis

I am trying to decide the appropriate model and perform power analysis. Suppose a data contains 2 groups (A,B) and 28 participants from each group. All of those participants are measured 15 times on ...
ksing's user avatar
  • 75
6 votes
1 answer
146 views

Why do the quantiles of a multivariate normal not work properly?

Let's say we have a multivariate normal distribution with two components. The two means $\mu_1$ and $\mu_2$ are both equal to 0 and the covariance matrix is a simple 2x2 square matrix with ...
Denzo's user avatar
  • 504
0 votes
0 answers
24 views

use of cpower function to estimate power of Cox-PH analysis

I am trying to calculate the power for Cox PH analysis using the cpower function in the Hmisc package. The following is sort of "post-hoc power", mainly to understand how the function works. ...
Yossi Levy's user avatar
1 vote
1 answer
28 views

How can I interpret non-significance in models built with subsetted data?

I am working on revisions, and it was requested that I re-run my models separately for males and females. When I do this, I lose statistical significance in some tests in females but not in males, but ...
HarD's user avatar
  • 207
2 votes
1 answer
109 views

Power of a Jeffreys test

The Jeffrey test is a tool used to evaluate evaluation of proportions, notably in the IRB Framework to evaluate the probability of default (see: ECB - Instructions for reporting the validation results ...
Lucas Morin's user avatar
  • 1,505
4 votes
2 answers
156 views

Confidence interval for proportion assuming binomial distribution when observed proportion is exactly 1 (or 0) [duplicate]

I have a simulation (a set of 200 iterations) which I used to compute a confidence interval for average statistical power. In binomial approximation, I know that the standard error is computed as $\...
Dovini Jayasinghe's user avatar
0 votes
0 answers
27 views

How to choose the sample size

How do I choose a sample size for a multiple factors irrigation experimental research? Here is what I mean, I want to compare the effect of precision irrigation and conventional irrigation on crop ...
Salami Chakour's user avatar
1 vote
1 answer
47 views

Sample Size Calculation for Gene and Protein Biomarker Study

The PIs are looking for biomarkers in pre and post-trauma individuals. I need to calculate a sample size for a microarray study of 10 genes and the top 5 up- and down-regulated proteins and ...
Tavaro Evanis's user avatar
0 votes
0 answers
28 views

Power Analysis for 8x8 factorial within-subjects design (required number of participants/samples)

for an experiment, I was asked to provide a minimum required number of participants using an a-priori power analysis. Thing is, I'm now really confused on how this is possible or even if it makes ...
WalterYellow's user avatar
2 votes
0 answers
33 views

Kruskal-Wallis statistical power

Given that the Kruskal-Wallis $H$-test has been computed, the epsilon-squared estimate of effect size can be calculated as $$E_R^2=\frac{H}{(n^2 - 1)/(n+1)},$$ the number of the compared groups, the ...
Andrea Gulli's user avatar
1 vote
1 answer
179 views

How to determine sample size needed for sufficient power to compare between levels of a fixed effect in a mixed model?

I ran a pilot study for a within-subjects experiment where the fixed effect of interest is categorical, with 4 levels. The research question involves comparing which levels of this IV can ...
MM812's user avatar
  • 11
2 votes
2 answers
74 views

What are some good resources on Bayesian unconditional power analysis, besides John Uebersax's article?

I refer to the article "Bayesian Unconditional Power Analysis" by John S. Uebersax (2007). I'd like to explore the subject further. I haven't checked yet the references that the Uebersax's ...
1 vote
0 answers
14 views

Could a study be considered to be underpowered if the effect size they detect is significantly below their predicted effect size? [duplicate]

I'm assessing a paper, the authors of which state that they recruited 3000 patients because that was what the power calculation suggested was necessary to detect a 5% difference at 85% power. They ...
user356816's user avatar
5 votes
2 answers
331 views

Should the Wilcoxon Rank sum test be used for testing the mean difference significance?

I was looking replicate the results of the paper DOI:10.3905/jpm.2014.40.3.087 (Exploring Macroeconomic Sensitivities: How Investments Respond to Different Economic Environments, Ilmanen Maloney Ross ...
Luca Dibo's user avatar
4 votes
1 answer
24 views

AB Testing with and without counterfactuals

My boss wants us to run a simple experiment where we manipulate the search results on our website. He is insisting that we must have counterfactuals for the results to be valid. The way I see ...
Daniel's user avatar
  • 221
2 votes
0 answers
44 views

Validating power analysis

I'm currently running some experiments in web. The main idea is to validate the impact in some conversion metrics. The typical funnel we have is: Traffic -> Signup -> some Activation event. Our ...
marz's user avatar
  • 187
2 votes
2 answers
80 views

If a study finds a statistically significant difference, but later is determined to be under-powered, does it matter? [duplicate]

Thanks for any answers in advance. This question presumes a study has detected a difference between 2 groups, and has determined this difference to be statistically significant. I'm reviewing a paper ...
user356816's user avatar
3 votes
1 answer
167 views

Calculating minimum sample size required for repeated measures linear mixed model using the simr package

I would like to use the simr package to calculate the smallest sample size needed to achieve $0.80$ power at the $0.05$ alpha level while accounting for a small ...
AHS's user avatar
  • 31
0 votes
1 answer
65 views

Required sample size for exact McNemar test and for phi coefficient (power analysis)

I aim to establish the required sample size for my study. In the main analyses, which are conducted to test my hypotheses, I am investigating two 2x2 tables. Specifically, I plan to perform an exact ...
Louis's user avatar
  • 1
4 votes
1 answer
68 views

How does G*Power calculate sensitivity in z-test for two independent proportions

I am trying to recreate sensitivity calculation in Python by applying the same methodology as used in G*Power z-tests > Proportions: Difference Between Two Independent Proportions > Sensitivity: ...
Eugene Krall's user avatar
0 votes
0 answers
47 views

How to: power analysis with log-normal distributed data

Assuming the following data: ...
ThePresident's user avatar
2 votes
0 answers
26 views

Is there any valid approach to adjust the statistical power taking the inflation of type 1 error into account?

I'm conducting some simulation analysis under different correlation structures where H0: absence of interaction component in the model. I computed type 1 error rate under the null hypothesis at some ...
Dovini Jayasinghe's user avatar
1 vote
0 answers
34 views

power analysis for cross-lagged panel model with random intercept using powRICLPM

I want to compute power analysis for cross-lagged panel model with random intercept using powRICLPM package (Related publication) Authors of the package did a great job and the power analysis seems to ...
David's user avatar
  • 11
4 votes
1 answer
152 views

Finding P-value and power of the Most Powerful Test

You observe a sample $X_1, \quad, X_{20}$ with the density $$ f(x, \vartheta)=2\left(x / \vartheta^2\right) I_{[0 \leq x<\vartheta]} $$ with an unknown parameter $\vartheta>0$, yielding $$ \min \...
Stats_Rock's user avatar

1
2 3 4 5
32