1
$\begingroup$

I am trying to figure out the right analysis & the required sample size for my study design. I am trying to look at outcome A of students' rating (Level 1) nested in a faculty (Level 2) nested in a university (Level 3).

I will most likely only have 2 universities, and 3 faculties each (6 faculties in total), so I am trying to see then how many students I will need in total.

However, it seems to me that the units for Level 3 (2 universities) and Level 2 (6 faculties) are too small to conduct a multilevel analysis, unless n of Level 1 is huge.

Could you please help me figure out if multi-level analysis is still feasible in my case?

Also, I cannot wrap my head around the power analysis and the exact required sample size for multilevel analysis in my case, it would be great if you could help me out!

Thanks!

Edit: I am only interested in a relation between students' rating (Level 1) and outcome A. I am adding Level 2 and Level 3 to see if I see any impact of those on the outcome (though not likely).

Edit 2: Given that I do not expect any influence of Level 2 & Level 3, could I also control for those variables in a regression analysis, instead of conducting a multilevel analysis?

$\endgroup$

1 Answer 1

2
$\begingroup$

Multilevel models can work in smaller samples when using appropriate estimation methods (restricted maximum likelihood [REML]) and standard error corrections (i.e., Kenward-Roger correction). But when you are dealing with less than 7-10 groups, multilevel models break down. Some very helpful recent work on this problem has been carried out by Daniel McNeish and colleagues.

Your intuition about adjusting for cluster membership via a set of indicators for each of the instructors is the way to go. Although note that you will not be able to add indicators for both instructors and schools as they are perfectly collinear. By knowing an instructor, you know what school they taught at (and vice-versa). If you wanted an article on the difference in this approach (vs. mlm) and when it might be more appropriate, see McNeish & Kelley (2018).

Edit: In terms of power analysis, once you move to the indicator approach, you can more or less conduct power to detect an effect within an OLS regression. There are general purpose tools for this, such as G*Power or other simulation tools.

$\endgroup$
1
  • 1
    $\begingroup$ Great, thanks a lot!! $\endgroup$
    – user240313
    May 14, 2021 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.