An inquiry into the quality of a statistical test by calculating the power - the probability of rejecting the null hypothesis given that it is false - under certain circumstances. Power analysis is often used when planning a study to determine the sample size required to achieve a nominal level of ...

learn more… | top users | synonyms

1
vote
1answer
21 views

ROC/AUC Confidence Interval

For a single ROC curve (with relevant AUC score), how can you calculate the confidence interval? (The data used to generate this ROC/AUC is available) Given my relatively limited background in this ...
1
vote
0answers
25 views

Power analysis for binomial data when sample sizes are different

I'm running an A/B test where I want to compare two different layout of a button and want to see if one layout lead to a higher Click-Through Rate with a certain statistical power. I want to use a ...
4
votes
2answers
70 views

Can I do a t-test Power Analysis for Unequal Size Groups which Produces 2 Different Minimum n's?

It's usually straightforward to do a Power Analysis to compute the minimum sample size, especially in R which is my preferred ...
1
vote
0answers
16 views

Power Analysis for Text Mining

I have a population of 6 million text files with which I want to perform sentiment analysis (and text analysis more generally). I will need to manually hand code a subset of these texts into positive ...
2
votes
0answers
16 views

When do sample size calculations need to adjust for clustering?

I am aware that sample size calculations for cluster RCTs must adjust for clustered data. I would like to understand better if such adjustment might be necessary for other study designs. For instance, ...
0
votes
0answers
13 views

Determine minimum number of observations to be checked manually

I'm trying to classify transactions (say credit card transactions) as good or bad without looking at it manually. (The only way to figure out whether a transaction is good or bad is by manually ...
1
vote
0answers
22 views

How to do a power analysis for an unbalanced mixed effects ANOVA?

I need suggestions for how to calculate the $n$ required for 80% and 90% power for >30% change from baseline ($T_0$) at $T_1$ or $T_2$, drug vs. placebo, 2:1 ratio; 20% CV in test; factors: subject, ...
0
votes
0answers
9 views

Power estimates for an interaction in a 2X2 factorial design with a proportion outcome

I am trying to calculate the power that I will have to detect an interaction between 2 different interventions in a 2X2 factorial design. We aim to have about 850 participants in each of the four ...
0
votes
1answer
11 views

How many samples are needed for a paired trial with multiple tries?

I'm having problem understanding how big sample size is needed for a situation where there are pairs in the before/after groups but these have all multiple data points. Problem Pose a situation ...
2
votes
1answer
39 views

Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
0
votes
3answers
71 views

Large sample size for t-test

I am working on a physics experiment, and I ran 500 runs to calculate a parameter. I want to use the t-test to see if my mean from 500 runs matches the actual value. u0: mean is actual value u1: ...
1
vote
0answers
39 views

Questions about G*Power

Since I found an insignificant result, I performed a power calculation in G*Power. But I am not entirely sure if I input the right number in some fields. My design is a 2(between Ss) x 2(between Ss) ...
5
votes
0answers
86 views

Minimum sample size for a dichotomous outcome

I have two questions. I am running an experiment where I am interested in determining the sample size required for a certain CI and error, where values range between $<1$ and $>-1$. However, I ...
0
votes
0answers
20 views

power analysis when mean and standard deviation are given

I have been asked to perform a sample adequacy test (i.e., I suppose, to verify what is the minimum difference that can be quantified between two grous) where a new sample (of which mu, sigma and ...
0
votes
0answers
26 views

Sample size calculation for paired t-test single arm using r

I am planning a study on alteration in AUC of a drug due to a surgery of gastrointestinal tract. It's a pre-post study. A group of patients will undergo blood sampling for AUC (area under curve) both ...
1
vote
0answers
20 views

Power of maximum likelihood parameter estimates for a linear model

The maximum likelihood parameter estimates for the linear model where $\Pr(Y|X\beta) \sim \mathcal{N}(0,\sigma^2)$ are: $$\hat{\beta} = (X'X)^{-1}X'Y$$ How do you compute the statistical power of ...
0
votes
0answers
24 views

Calculating sample size. Repeated measures within subject

I'm trying to calculate the sample size for a repeated measures within subject experiment. I'm going to test the subjects under 4 conditions and for each condition 6 measurements are taken at ...
0
votes
0answers
9 views

G* power for ANOVA repeated measures

I'm using G*Power to obtain an estimation of the sample size required. I have a 3×3 within-subject design (i.e., two factors,with 3 levels each). I used the partial eta squared of one of my ...
1
vote
0answers
24 views

What's the sample size required to confirm that two treatments are identical?

Goal: Determine if a machine is defective Say I am going to give the same treatment to a group of people. I have a machine that will randomly split them into $2$ groups before giving the same ...
1
vote
0answers
19 views

Calculating power for a survey to compare different metrics

I am currently planning a survey. The data will be used to test which of three metrics best captures some "true" underlying characteristic of the respondents. Getting data on the "true" underlying ...
0
votes
1answer
32 views

Designing Frequentist A/B Experiments

What parameters should one pre-specify before running an experiment? (e.g. a two-sample, one-side, t-test) My understanding is that on establishes: Null hypothesis Maximum Significance level ...
3
votes
0answers
27 views

Estimating Sample sizes for Null Hypothesis Testing

Wikipedia has the following table for computing sample sizes:                                       ...
1
vote
0answers
28 views

Is the variance of the residuals of a linear regression useful for estimating experimental sample sizes?

I have a data set of $y$ values that is not particularly normally distributed. However, the $y$s do partially depend on several other parameters. A linear regression model $y=c+\mathbf{ \beta ...
1
vote
0answers
24 views

power for estimating treatment heterogeneity in RCT

Could someone point me to a good resource for estimating power for detecting treatment heterogeneity via interactions in MR? Let's assume that individuals will be randomized to two study arms. To ...
3
votes
0answers
24 views

Power in proteomics?

Grants often require power analysis to support a proposed population size. In proteomics (and most -omics), there are 100's to 1000's of features/variables measured across 10's of samples (maybe 100s, ...
2
votes
0answers
51 views

Sample size requirements when studying total population

I work for a fundraising organization and we want to implement an A/B test to determine if a particular donor recovery technique is working. The population under study is our lapsed donors. These are ...
0
votes
1answer
14 views

How to perform power analysis for repeated measures with observation error?

I am interested in producing two raster maps of the same areas but separated in time. I would like to take the average value of all pixels in each map and test if there is a difference in that mean ...
0
votes
0answers
22 views

Is unpaired t-test appropriate for 2-level random cluster sampling? Also, how does this affect the power of the test?

I'm working on power/sample size calculations for a non-profit and want to make sure my logic is correct, as I haven't worked with a situation quite like this before! This is a bit of a long post, as ...
0
votes
0answers
25 views

Power analysis in meta-analysis: Compute the maximum significance needed by an additional sample for meta-analysis to reach genome-wide significance

I have an association study analysis (GWAS) that does not reach genome-wide significance (max p-value < 10E-5), and there is a potential additional sample (so with known N and known effect size/SE ...
5
votes
1answer
35 views

Power analysis for post hoc test of ANOVA with many groups

In both G*Power and the R pwr package, estimated sample size required per group decreases as the number of groups increase. This seems somewhat counter-intuitive. For a toy example, assume that I ...
5
votes
1answer
81 views

Power when using binomial data versus continuous data

I have a question about something that is probably very basic to statistics but I feel I don't fully understand I've found that generally it is much harder to get high power (smaller) confidence ...
1
vote
0answers
42 views

Power Analysis: Comparing GLM to binomial proportion test

I am trying to run a power analysis using simulated data to compare the power of a glm versus a binomial proportion test to detect differences in proportions. For example, suppose you have some ...
0
votes
0answers
30 views

Monte Carlo Based A-Priori Power Calculation for Logistic Regression

I have been wanting to get a Monte Carlo based power calculation working for Logistic Regression cases for a little bit. I have put together a workflow with some assisstnace and I wanted to ask the ...
6
votes
1answer
141 views

Required number of simulations for Monte Carlo analysis

My question is about the required number of simulations for Monte Carlo analysis method. As far as I see the required number of simulations for any allowed percentage error $E$ (e.g., 5) is $$ n = ...
1
vote
0answers
31 views

Monte Carlo for DOE Power Calculation When Performance is Unknown

I have been wanting to get away from using power calculation tools (JMP, Lenth) in favor of Monte Carlo techniques to determine the a-priori power to observe factor level effects in a DOE. I am ...
0
votes
1answer
32 views

Can Inferential Statistics be carried out in a Retrospective Case Study?

Just wanted to check with regard to a retrospective case study, if various data is collected for a set of patients on different variables can this be interpreted with inferential stats tests? Even if ...
2
votes
0answers
29 views

Retrospective Case Study

I am carrying out some research and am a novice to statistics, having done a 2 week module primarily focusing on parametric data and experimental design. However, for my research I am doing a ...
2
votes
2answers
113 views

Website (A/B or Multivariate) testing, Statistical significance not lining up with sample size

So I have setup testing code on a website, our conversion data is blocked per day and fed into R. This is the R code that I am using to calculate the "confidence" ...
2
votes
0answers
23 views

Noncentrality Parameter - what is it, what does it do, what would be a suggested value?

I have been trying to brush up on my stats knowledge, especially in relation to Sample size determination and Statistical Power Analysis. But it seems that the more I read the more I need to read. ...
3
votes
0answers
77 views

How to modify post-hoc power analysis simulation to perform a-priori power analysis?

I am interested in doing a sample size calculation for exacerbation rates. I am drawing my data from a negative binomial distribution and my test statistic is based on a negative binomial glm. ...
1
vote
2answers
51 views

Power calculation for a test of trend over time

This paper (open access) Babies born at the threshold of viability: changes in survival and workload over 20 years (table 2) presents data for survival of extremely premature babies over the last 20 ...
1
vote
0answers
39 views

How to compare the price of two sets of business transactions?

A business transaction can be defined as a function of 6 independent categorical variables (representing the customer and the product). Sales representatives are allowed to charge whatever price their ...
1
vote
1answer
67 views

How can I find a level $\alpha$ most powerful test?

Let $X_1,X_2,\dots ,X_n$ be a random sample from a distribution with pdf $f(x,\theta)$. Find a level $\alpha$ most powerful test of $H:\theta=\theta_0$ against $K:\theta = \theta_1$ when ...
0
votes
0answers
32 views

Power analysis in a linear model - pooled vs individual samples

I'm interested in how many samples are required to see a relationship between allele frequencies (proportions of alleles) and environmental gradients, such as temperatures and water quality. ...
1
vote
0answers
91 views

How to compute sample size to detect a small change?

Suppose I wish to calculate the sample size needed to detect a one percent change in a proportion that is normally at 35%. I don't mean 35% to 36%, I mean 35% to 35.35%. Here is the calculation in R ...
1
vote
0answers
117 views

Sample size / power calculation for logrank survival test

I have really little idea of survival analysis. I am aware that logrank is a special case of Cox' proportional hazards model, and that tons of R packages and scripts address the problem of power / ...
7
votes
2answers
111 views

In a meta-analysis, how should one handle non-significant studies containing no raw data?

Let's say that I'm conducting a meta-analysis, looking at the performance of group A and group B with respect to a certain construct. Now, some of the studies that I'll come across will report that no ...
5
votes
2answers
82 views

SD larger than mean, non-negative scale

I was given an article reporting a study very similar to one my lab wishes to run. But, I noticed that for the variable of interest, Duration, the SDs are larger than the mean...since this is ...
1
vote
0answers
71 views

Sample size calculation for logistic regression using simulations

I want to find the sample size for logistic regression where I have a covariate with 15 levels and the covariates interacts with time, which means that the effect of the covariates is different for ...
1
vote
0answers
31 views

Joint power calculation for two correlated test statistics

I have a situation where I want to perform two tests, where the null hypothesis is rejected if test 1 is significant and test 2 is not significant. If the two tests are independent then I think the ...