I will perform a behavioral experiment involving a 2 by 2 design. The experiment will be completely within subjects, i.e. each subject will perform each of the possible combinations of factors once, so that I'll have 4 measures for each subject.
I want to analyze my results with a mixed effects linear model that will have my two factors as fixed effects and a subject-specific intercept as random effect.
My hypothesis is that neither the main effects nor the interaction of the factors are significant, i.e. subjects performance will be comparable in all conditions. Especially in psychology, it is extremely difficult to publish a negative result (especially if it is about a much hyped topic), so I have to be sure that sample size doubts will not came up later on.
I have looked around to find methods to calculate the sample size for a mixed effect model, but I haven't found anything that really satisfy me.
The first thing that I've looked into is G power (http://www.gpower.hhu.de/): this software can calculate a-priori sample size for many different statistical tests, but the closest thing that I found is a repeated measure ANOVA with what they call a "within factor effect". This option only allow for a number of group >= 2, that it's clearly not my case.
I've then turned to Cross Validated, and the closest answer that I found is this one (Sample size calculation for mixed models). However, the lmmpower function referred to in this answer seems to be very specific about the kind of design it accept: a longitudinal design where the effect of interest is the rate of change over time, as stated in the help
"Pilot estimates are assumed to be from an appropriate "placebo" group and the parameter of interest is assumed to be the rate of change over time of the outcome."
In the same answer is mentioned that you can always use simulation to calculate your sample size, and I am trying to wrap my head around the question of how I can use simulation in the context of my experiment.
Here what I have thought:
1 - I need to figure out what is the "minimum meaningful difference of the means", i.e. what is the minimum difference beyond which I can state that my conditions do not differ
2 - I need to create simulated data (sample size = N) in which the two conditions differ (on average) of the value found at point 1
3 - I need to fit my model to the simulated data and register if I could find or not a significant difference
4 - Repeat point 2 and 3 K times and calculate the proportion of successful detecting of the difference
5 - Repeat point 2,3 and 4 with different Ns
6 - Find for which N I could detect the difference let's say 80% of the time
Now, my questions are:
1) Does what I have exposed have any sense at all ?
2) I can understand the way in which I can find the "minimum meaningful difference of the means" between the two levels of the two factors, but what about the interaction ?
3) How can I reproduce the clustering seen when dealing with repeated measures (and do I need to reproduce it all, even if I guess the answer is yes) ?
4) How big should K be ?