# Power calculation accounting for test reliability

I have a patient group and a control group. Each group will take a medical test. I would like to estimate the required sample size (power 0.8, alpha 0.05) if the effect size (cohens d) between groups (as pertains to between group differences for results on this test) is 0.5.

This part is straightforward - and I would use a program like G*power to perform the calculation.

The issue that I am unclear about is how to factor in the reliability of the test? For example I have data that the intraclass correlation coefficient obtained in test-retest study is 0.6.It seems clear that a less reliable test will have a lower chance of detecting group differences but I am unsure how to correctly quantify this.

Any help would be much appreciated

Rob

• digitalcommons.wayne.edu/cgi/… This paper goes someway to answering the question., giving tables that would allow me to estimate. One further question - is the intraclass correlation coefficient a suitable parameter to plug into the "reliability" part of the tables? Rob – RobMcC Jan 26 '17 at 13:48
• Your estimate of effect size should reflect the level of reliability of your measure(s). Do you have reason to think it does or does not? – Joel W. Jan 26 '17 at 14:05
• Dear Joel - yes this is a new medical test so their is no pilot data to base the effect size on that would account for the reliability of the measure. The effect size is instead being estimated a priori based on the expected underlying (true) biological differences between the groups – RobMcC Jan 26 '17 at 14:12
• Based on y=r*x, my guess is that the expected observed effect size will be r times the true effect size, where r is reliability. I look forward to a definitive answer to your question. – Joel W. Jan 26 '17 at 15:09