0
$\begingroup$

I am trying to conduct a power analysis on a multilevel longitudinal model (pre-existing household panel dataset) and am having trouble figuring out how to do it (or if it is even needed in the first place, however I am assuming that I will be asked the question by reviewers if I do not include it). I am using R for all analysis (lme4 package) and would therefore prefer an R solution if possible (but it doesn’t have to be).

Below is a summary of the model.

Outcome DV

  • Job satisfaction measured over time (up to 10 time points)

Level 1 (within-person) variables:

  • Time
  • Personality trait 1
  • Personality trait 2
  • Personality trait 3
  • Personality trait 4
  • Personality trait 5
  • Employment status (part-time or full-time, coded as 0 or 1)
  • Monthly gross income

Level 2 (between-person) variables:

  • Time (the between-person component of the level 1 variable)
  • Personality trait 1 (the between-person component of the level 1 variable)
  • Personality trait 2 (the between-person component of the level 1 variable)
  • Personality trait 3 (the between-person component of the level 1 variable)
  • Personality trait 4 (the between-person component of the level 1 variable)
  • Personality trait 5 (the between-person component of the level 1 variable)
  • Employment status (the between-person component of the level 1 variable)
  • Monthly gross income (the between-person component of the level 1 variable)
  • Gender
  • Average age

Cross-level interaction:

  • Time (level 1) x Personality Trait 1 (level 2)

The model has job satisfaction as its outcome variable where I am interested in modelling the growth in job satisfaction as a consequence of between-person (level 2) extraversion (hence the cross-level interaction). Individuals serve as the level two clusters (N = 7,275) and observations of time serve as the primary level one predictor (N = 23,974, unbalanced such that different individuals have different N observations). All other variables are covariates and for all time-varying covariates I have split them into their within-person and between-person components (hence the double-up at level 1 and level 2 for all variables but age and gender).

Any help would be appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

A couple of points:

  • Power analysis for an existing dataset does not really tell you much. It basically gives you the same conclusion as the p-value if you analyze the data. For more discussion on this, check this post.
  • If you still want to do it, then given the complexity in your design I would say that the only viable approach is simulation. That is, you simulate, say $B = 1000$ datasets from your model based on the postulated design, you fit the model in each dataset and you perform the test of interest and keep the p-values. The proportion of times the p-value was significant at a give level will be your power.
Improve this answer
$\endgroup$
4
  • $\begingroup$ I agree with your first point but was not aware of that post - very helpful, thank you! That said, I would still like to know how to conduct such a test for my model (or future models). Any links/references to assist with this would be appreciated (the post you linked doesn't seem to cover MLMs). $\endgroup$
    – aspark2020
    Commented Nov 27, 2018 at 6:30
  • $\begingroup$ You mean a link/reference on how to simulate from a mixed model? $\endgroup$
    – Dimitris Rizopoulos
    Commented Nov 27, 2018 at 8:28
  • $\begingroup$ Yes, so basically I would like to know how to implement in your second point (although, I appreciate that this specific question may be better suited to Stack Overflow). $\endgroup$
    – aspark2020
    Commented Nov 28, 2018 at 23:45
  • $\begingroup$ Check this post for a piece of code that will need to be suitably adapted though: stats.stackexchange.com/questions/376570/… $\endgroup$
    – Dimitris Rizopoulos
    Commented Nov 29, 2018 at 4:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.