Questions tagged [statistical-power]
Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.
1,530
questions
0
votes
0
answers
7
views
semPower: Accounting for Clustered Standard Errors in SEM Power Analysis with Fixed Effects Model
I am planning to analyse clustered data with a structural equation model and as I am planning on submitting my project as a registered report, I am first trying to do an in-depth power analysis (it is ...
2
votes
0
answers
23
views
Logistic Regression Power Analysis with Multiple Predictors
So I'm working on my dissertation and I'm trying to figure out a binary logistic regression power analysis to show how many participants I should collect. My proposed analysis is as follows and the ...
- 21
1
vote
1
answer
62
views
sample size calculation for generalized least squares regression
I want to calculate the sample size for a multiple linear regression with correlated residuals, specifically using generalized least squares. For OLS, it appears that there a various different rules ...
- 333
1
vote
0
answers
29
views
G*Power repeated measures ANOVA within-within interaction
Suppose that I have 4-by-6 factorial design where both factors are within-subject variables, and I want to perform a repeated-measures ANOVA on the data. Using GPower, I calculate the sample size ...
1
vote
1
answer
22
views
Is a power analysis appropriate when the predictor variable is bounded between 0 and 1?
I am doing a power analysis (PA) to find the minimum sample size to detect a difference between two groups (with acknowledgment of the theoretical problems involving using p-values, etc.).
The mean ...
- 67
0
votes
0
answers
8
views
High Type 1 Errors in Power Analysis by Simulation for Null Effect with Multilevel Ordinal Regression with Cumulative Link Model
I am doing a power analysis for an experiment where we will fit a cumulative link model for ordinal regression. But when I do the power analysis, my power for the null effect is between 20% and 30%. ...
- 155
0
votes
0
answers
12
views
Statistical consequences of random sampling or longitudinal (i.e., repeated) sampling
I need to assess the effect of different treatments on a crop. I am using a completely randomized block design with 4 treatments and 4 replicates for each treatment. The variables I need to measure ...
- 1
3
votes
2
answers
93
views
Test if a correlation is bigger than a specified value?
I have calculated the correlation $r$ between $X$ and $Y$. I'd like to test $H_0: r \leq \rho$, $H_1: \rho < r$.
For the case when $r = 0$, I can apply the transform $t = \rho \frac{\sqrt{n-2}}{\...
- 41
1
vote
0
answers
25
views
What is the minimum number of data points/observations required to use Theil-Sen?
I am working on an algorithm which requires estimating trend magnitudes of data points. I have been told to use Theil-Sen as it is more robust to outliers and it is non-parametric. As users will be ...
- 11
2
votes
1
answer
42
views
Sufficient Sample Size for Three-way Interaction from Observational Data (2x2xContinuous)
I'm wondering how one determines the sample size needed to sufficiently analyze a three-way 2x2xcontinuous (logistic) regression interaction using observational data (association study)? Any ...
- 123
0
votes
0
answers
35
views
How to identify a one vs. two-sided test (power analysis on proportions in R)
I have an oddly worded assignment for a power analysis on (paired) proportion data and I'm not sure if it matches the description of a one- or two-sided test. More specifically, I need to "see if ...
- 1,041
0
votes
0
answers
20
views
Power analysis for cox-ph model for 8 (correlated) groups?
I am investigating covariate X for 8 cancer types which all have different survival rates and will possibly show different impacts of other covariates like age, sex etc. Therefore I will run 8 ...
0
votes
0
answers
24
views
Converting a continuous metric to a proportion for increased power
I run online A/B tests and I guess I have some basic stats background, but I've encountered a situation where I don't feel confident about.
We plan to track page load time in each experiment to make ...
- 131
0
votes
0
answers
33
views
How to calculate the sample size when compare rate using Poisson regression?
I have a question regarding the sample size calculation if I want to compare rates using Poisson regression. I want to compare the rate of book A to the rate of book B. The rate of book A is defined ...
- 41
0
votes
0
answers
16
views
How big a sample do I need to estimate a percentage?
I have a population of 1000000 data points without labels, and I would like to take the smallest possible sample from that population for manual labeling to estimate with 95 % certainty that the ...
- 131
1
vote
0
answers
29
views
How big a sample size do I need to confidently calculate if my "False Negative Rate" has changed over time?
Problem
I have a classification model, and I would like to know if the distribution of its predictions stay stable over time (e.g. the False Negative Rate).
The model
The model is a binary classifier ...
- 131
5
votes
1
answer
237
views
What is the substantive meaning of one statistical test is more powerful than another?
There are some research claims that one statistical test is more powerful than another. For example, a highly cited study states:
Results show that the Shapiro-Wilk test is the most powerful ...
- 408
1
vote
0
answers
54
views
Power analysis on paired proportions data
I'm interested in running a two-sided power analysis on paired proportions data and hope to use the stats:power.prop.test or pwr::pwr.2p.test function, but not sure if the functions assume paired or ...
- 1,041
0
votes
0
answers
42
views
calculate effect size from regression table
I am working on power analysis for a project. I want to benchmark effect size from another study. How can I find the effect size (Cohen's d) just by looking at the regression table below? For instance,...
- 111
1
vote
0
answers
12
views
Sample size calulation for glmm models
I am trying to calculate the sample size for my next experiment.
I have done a pilot study with 16 participants and made a mixed model using lme4 as below.
...
- 11
1
vote
0
answers
32
views
Difference-in-Difference: Statistical Power Test R
I am planning on running a natural experiment (difference-in-difference) on some UK Local Authority level mean obesity prevalence data. Sample size is limited by a number of factors and it is ...
1
vote
0
answers
19
views
Hypothesis test for treatment with opposite signed effects across subgroups
Imagine a treatment that, if it has an effect, is likely to
affect two groups in opposite directions. As a concrete example, suppose
I am testing the effect of a Republican politician endorsing a ...
- 268
1
vote
0
answers
14
views
Estimate minimum detectable effect using simulations [closed]
In a study investigating a time-to-event outcome in a propensity score matched sample, the authors state the following in their methods: "Where no evidence of difference between the compared ...
- 11
0
votes
0
answers
10
views
Simulate competing risks
I am trying to simulate competing risks data for a power analysis. Based on historical data, we know that after 12 months:
40% had experienced the event of interest
20% had experienced the competing ...
- 1,840
1
vote
1
answer
70
views
Power Analysis for Difference in Differences Regression
How would you approach conducting a power analysis to estimate how large of a sample you would need to conduct a difference in differences estimate?
For example, for a T-test, I would just set effect ...
- 11
0
votes
0
answers
42
views
Power analysis for correlated data points
I want to using simulation to determine the effect size that could be detected with my sample size. What makes the simulation a bit complicated is that my dataset is about species traits. It means ...
- 1
0
votes
0
answers
44
views
Power/sample size estimation when comparing two AUCs (area under the curve)?
Are there any R functions (or other free software) for calculating power/sample size needed to compare 2 AUCs (area under the curve)?
Specifically, suppose you want to fit 2 different models to a ...
- 1
0
votes
1
answer
169
views
Power analysis in R using pwr for glm
I am trying to conduct a post-hoc power analysis based on the generalised linear model I used in my study. I'm trying to determine what the power is so that I can use it to compare against another ...
- 15
0
votes
0
answers
64
views
Using regression predictions to evaluate randomised trials - good idea or not?
I need to evaluate the results of an experiment using a black-box ML model.
Normally, if I have a series of outcomes from the control group $ X_i $ and the test group $ Y_i $ I would just do a t-test ...
- 115
0
votes
0
answers
15
views
How to estimate sample size needed for a planned comparison between two groups in a between-within ANOVA?
I have a 3X3 repeated measures ANOVA with 3 groups and 3 measurement timepoints. In addition to the omnibus effect, I am interested in comparing the difference in change within two timepoints between ...
- 313
0
votes
0
answers
13
views
Power of mixed effects model vs ANOVA repeated measures within-between interaction
I'm trying to estimate the power of a 3X3 longitudinal mixed effects model with 3 groups, 3 measurement timepoints, and participant ID as a random intercept.
I know that using the "ANOVA repeated ...
- 313
1
vote
1
answer
26
views
Power analysis using bootstrap -- calculate power
I wish to conduct a power analysis using the bootstrap method to find the difference in means of two groups. I wonder how we can calculate the "power" if we do not know in our data, the null ...
- 11
1
vote
1
answer
50
views
Power analysis for minimum effect tests and good enough range hypotheses
Power analyses allow one to determine the minimum sample size required to detect a certain effect (when there is one)
To calculate power, one determines the minimum effect one wants to detect
With ...
- 319
1
vote
0
answers
19
views
How rigorously can boundary conditions on statistical power of stratified analysis vs. pooled analysis be defined?
I spend a lot of time thinking about conditions under which various kinds of analyses perform well. To some degree I am hoping NOT to specify too many specifics here (because part of what I am ...
- 284
0
votes
0
answers
21
views
Relation between the sample size with the powers of Kolmogorov Smirnov, Cramer Von Mises and Anderson Darling?
I want to know the mathematical relation between the sample size with the powers of 1. Kolmogorov Smirnov, 2. Cramer Von Mises, and 3. Anderson Darling.
I have seen many posts and other websites ...
4
votes
1
answer
104
views
Is the Neyman-Pearson paradigm for hypothesis testing adequate in the social sciences, and if not, is calculating a priori power still adequate?
I'm reading about Fisherian vs. Neyman-Pearson paradigms relative to hypothesis/significance testing. (NB: I'm new to this and self-studying it, so I'd be glad to be corrected if I misinterpret ...
- 43
3
votes
1
answer
34
views
Does the power of a one-tailed test with $\alpha=0.05$ approx equal the power of a two-tailed test with $\alpha=0.1$?
I have found this quote from a (to me obscure) power calculation software:
For a given effect size, sample size, and alpha, a one-tailed test is more powerful than a two-tailed test (a one-tailed ...
1
vote
1
answer
70
views
Power analysis for simple LMER model in R from scratch
I'd like to compute power for some rather simple lmer models. I am aware of some packages like simr but the tutorials are not easy to follow because they ...
0
votes
0
answers
22
views
mixed within-subject/between-subject design: how to build a model for power analysis
We have the data for three different treatments (A,B,C) in between-subject design. The means are correspondingly (X,Y,Z).
Now because reviewers insist, we need to re-run it in such a way that each ...
2
votes
2
answers
63
views
Conceptually, what is the bias of the standard error of an estimator?
I'm reading Muthén and Muthén (2002) to learn how to use Monte Carlo simulation to estimate statistical power in regards to the coefficients of a model that is linear in its coefficients.
I understand ...
2
votes
1
answer
33
views
How do I explore of the feasibility of using difference in days spent at home to assess outcomes after surgery?
I have done a observational study testing a hypothesis something like this "Days alive and out of hospital for one year after surgery is more than one year before surgery". My data looks ...
- 21
2
votes
1
answer
72
views
G*Power ANOVA repeated measures within-between interaction
I'd love to get a sanity check on a sample size calculation I'm running in G*Power. I'm planning a study with 2 groups and 2 measurement points, aiming for a f of 0.10 because, to my understanding, it'...
- 313
2
votes
1
answer
71
views
How to simulate multilevel data based on with- and between-group correlations?
I want to conduct a power simulation for my research with crossed random effects model in the R environment. Specifically, in the pilot study, each participant randomly rates 10 stimuli out of a ...
- 31
0
votes
0
answers
41
views
Statistical power and sample size determination using Kruskal-Wallis test
I have data that represents treatments applied to a given number of subjects in two or more groups. After each treatment, the subjects in each group are appraised to be of quality 1-5 and the number ...
- 153
0
votes
0
answers
30
views
Do the sample sizes for the Kruskal-wallis test need to be similar?
I have a dataset with around 42000 samples. For each sample I have a continuous variable "abs_error" (coming from a experiment ran before, not relevant here) and the subject's race and sex. ...
- 1
0
votes
0
answers
19
views
UMP test of $H_0:\theta=1$ vs $H_A:\theta\neq1$ for Beta($\theta,\theta$)
Let $X_1,X_2,\dots,X_n$ be iid Beta($\theta,\theta$) samples. Is there a UMP level $\alpha$ test of $H_0:\theta=1$ vs $H_A:\theta\neq1$?
We first test $H_0:\theta=1$ vs $H_A:\theta<1$. The ...
- 125
1
vote
1
answer
47
views
Bootstrap hypothesis test and Power Analysis: should I build my Power Analysis on t-test or Bootstrap, if hp testing is done through Bootstrap?
I'm using the bootstrap percentile method to frequently do hypothesis testing (two-tailed test for independent sample where H0 marketing KPI of group A - marketing metric of group B = 0; H1 is ≠ 0) ...
0
votes
0
answers
32
views
How to calculate power in Cochran's Q test
I want to know how to calculate power in Cochran’s Q test. I performed an experiment designed within-participants. Participants respond to one question in each three conditions. The number of ...
- 1
2
votes
2
answers
120
views
Does it make sense to do the power analysis if one of the sample proportions is 0%? [closed]
Can we have any sample size if one of the sample proportions is 0. I mean even the slightest difference detected between the two groups would be because of the treatment in this case, and hence I do ...
- 21
0
votes
0
answers
42
views
How to calculate effect size to to use Power Analysis to get minimum sample size when standard deviation is not available (digital marketing AB test)?
TLDR: I can't find a way to calculate the effect size needed to be inputted to Power Analysis to get the minimum sample size needed to 90% statistical significance as I can't calculate standard ...