3
$\begingroup$

I am planning a study that will use a hierarchical design. The study has participants engage in a decision-making task, wherein they make 40-50 decisions between 'Choice A' and 'Choice B' (I won't bore you with the details). Each decision a participant makes will be made in respect to varying "environmental" variables, such as the success of past decisions and certain aspects of the study design that make 'Choice A' or 'Choice B' more optimal. I am planning to have at least 250 participants, each making about 40 decisions (10 000 data points, then). The outcome variable is dichotomous (1 = Choice A; 0 = Choice B). The predictor variables at the participant-level will include age, gender, and income. The predictor variables at the decision level will include 'success of past decisions' and 'expected value of each decision'. The scarcity of predictor variables is due to us testing a new experimental paradigm ("bare bones" study).

In reading Ronald Heck's book on multilevel modeling, I get the sense that in MLM/mixed-effects designs that have individuals nested in organizations (rather than decisions nested in individuals), a level 2 sample size of 250 organizations and level 1 sample size of 40 individuals per organization will be sufficient. However, I am hesitant to apply this to my study when determining sample size.

Additionally, I am not specifically reading Heck's text about categorical outcomes, so I am wondering if sample size requirements for MLM with a continuous outcome is far different from sample size requirements for MLM with dichotomous outcome.

To wrap up my speech: is there anything inherent in having an MLM with participants at level 2 and their decisions at level 1 that would make my sample size requirements larger than 250 level 2 clusters with 40 observations in each cluster?

I'm aware there is probably more information needed for this decision. An approximate answer will do just fine, but if there is other info that is absolutely necessary, then I'm happy to do my best to provide it.

This is my first experience with multilevel modeling, so please excuse my ignorance.

Thank you in advance.

$\endgroup$
3
$\begingroup$

I would be extremely wary of any rule of thumb such as this:

a level 2 sample size of 250 organizations and level 1 sample size of 40 individuals per organization will be sufficient.

For one thing, sufficient for what ?

For another, a lot depends on the correlations within clusters, the higher the correlation, the lower the effective sample size.

In my opinion the best way forward here is to do a power analysis by simulation. If you use R, then the simr package would be a good place to start:

https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/2041-210X.12504
https://humburg.github.io/Power-Analysis/simr_power_analysis.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.