0
$\begingroup$

I have a multilevel model in which my response variable consists of a variable that is computed as the mean value of a set of observations (scored from 1 to 7). However, when I compute my response variable as the median or mode, some of the interaction effects between my predictors shift from significant to non-significant (e.g. group x predictor, group being binary and the predictor being able to have a value ranging from 1 to 7 each time). This in both directions (e.g. some interaction effects are only significant when using the mean, whereas other are only significant when using the median or mode). Main effects are uneffected.

I find this puzzling and have been trying hopelessly to figure out what’s going on. Any tips are welcome

Improve this question
$\endgroup$
4
  • $\begingroup$ Pretending that Likert data are numerical, so that sample means express something useful, is controversial; you should ponder whether you should be using sample means at all in your study. Likert data provide ordinal categories, for which medians and modes are clearly relevant. // Are significant interactions crucial to making sense of your data? $\endgroup$
    – BruceET
    Commented Aug 16, 2020 at 21:21
  • $\begingroup$ What the research is about is figuring out how preprocessing choices affect statistical results, and as they seem to have an effect, I would like to be able to explain why that is so. The significance or insignificance is hence not that important, but the fact that it changes based on these different choices is what makes it interesting :) Is there some literature you would recommend ? $\endgroup$
    – Jeroen
    Commented Aug 16, 2020 at 22:21
  • $\begingroup$ So you have a set of Likert-scaled items that you either compute a mean, median, or model to represent the scale? Have you considered an even more principled approach such as factor analysis or a full SEM with the measurement model for the outcome included? $\endgroup$
    – Erik Ruzek
    Commented Aug 17, 2020 at 13:37
  • $\begingroup$ The sets of items I take the mean/median/mode of have been used in research for ages to construct the factor at hand (always as the mean though), and I hence did not consider this. I ran a SEM at the within and between person level and found that at the within person level (same items are measured multiple times within one individual) the items do not load well, whereas they do load well at the between person level. This occurs for both my groups. Not yet sure what to make of this. $\endgroup$
    – Jeroen
    Commented Aug 20, 2020 at 7:50

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.