Determination of effect size for a repeated measures ANOVA power analysis

Question

This is a follow-up to the repeated measures sample size question.

I am planning a repeated measures experiment. We record energy usage for 12 months, then give (a randomly assigned) half of the customers continuous information about their energy usage (perform the treatment), and record their energy usage for another 12 months. A similar study performed in the past showed a 5% reduction in energy usage.

I want to estimate the required sample size using $\alpha=0.05, \beta=0.1$. G*Power 3 has a tool for repeated measures power analysis. However, it requires two inputs that I am not entirely familiar with:

• $\lambda$ - the noncentrality parameter (How do I estimate this?)
• $f$ - the effect size (I believe that this is the square root of Cohen's $f^2$)

According to Wikipedia's effect size page:

Cohen's $f^2= {R^2_{AB} - R^2_A \over 1 - R^2_{AB}}$ where $R^2_A$ is the variance accounted for by a set of one or more independent variables $A$, and $R^2_{AB}$ is the combined variance accounted for by $A$ and another set of one or more independent variables $B$.

However, my expected 5% change in energy consumption does not tell me how much variability will be explained. Is there any way to make this conversion?

If you know of a way to do this power analysis in R, I would love to hear it. I am planning to simulate some data and try using lmer from the lme4 package.

Answer 1

Assuming you are going to average the first 12 months to form a baseline measure and the second 12 months to form as a follow-up measure, your problem reduces to a repeated measures t-test.

G*Power

You might want to check out the following menu in G*Power 3: Tests - Means - Two Dependent Groups (matched pairs). Use A priori, $\alpha=.05$, Power = 0.90. Use the Determine button to determine effect size. This requires that you can estimate time 1 and 2 means, sds, and correlation between time points.

If you know nothing about the domain, based on my experience in psychology, I'd start with something like

M1 = 0, SD1 = 1, SD2 = 1
correlation = .60

This means that M2 is basically a between subjects cohen's d.

You could then examine a few different values of M2 such as 0.2, 0.3, ... 0.5, ... 0.8, etc. Cohen's rules of thumb suggest 0.2 is small, 0.5 is medium, and 0.8 is large.

R

UCLA has a tutorial on doing a power analysis on a repeated measures t-test using R.

Side point

As a side point, you might want to consider having a control group.