3
$\begingroup$

I've recently started analysing the data for a project using linear mixed models but am not sure how to deal with crossed and nested factors at the same time.

In my study, each participant reported two events (Event A: a negative personal experience they had & Event B: a negative news story they learnt from the media). In other words, each participant reported two negative events that had happened already, so there won't be cases wherein a participant shared an event that did not happen. Also, these events were independent of each other.

After 24 hours, we asked participants whether they discussed each event with someone else (Status: shared or not shared). A participant can have discussed Event A but not Event B (or vice versa), both events, or neither of the events.

EDIT 1: Then, we asked them to rate how much they thought their feelings about each event had changed during the 24 hours. In other words, each participant provided 2 ratings, one per event.

enter image description here

The dependent variable (DV) is perceived emotional change. I want to test:

  1. whether there's a main effect of Event on the DV
  2. whether there's a main effect of Status on the DV
  3. whether there's an interaction between Event and Status on the DV (e.g., let's say sharing leads to greater emotional change than not sharing, but this difference may be significant only in the case of Event A but not B)

I initially structured the model in this way in Python:

smf.mixedlm("affect_change ~ Event*Status", data, groups=data["participant"]).fit()

Or, in the R version: affect change ~ Event * Status + (1 | participant)

However, I realised that my study design is probably crossed and nested: Each level of Event (A, B) occurs with each level of Participant (that is, every participant reported both events), hence crossed. However, each level of Status (Shared, Not Shared) occurs with only either level of Event (that is, Event A was either shared or not shared, and same was Event B), hence nested.

Is my model structured correctly such that it reflects my study design and is able to test my predictions?


EDIT 2: I have another variable, happiness, that I measured before the 24-hour period and after it for each event. To be more specific, we asked participants to rate how happy they were about each event right after they reported it and asked again after they indicated whether they shared each event. enter image description here

How should I account for the additional effect of Time? That is, besides the effects of Event and Status (and possibly their interaction), I also want to check for the effect of Time (and if it interacts with the other two predictors) on happiness.

Thank you for reading my question.

$\endgroup$
2
  • $\begingroup$ It's not quite clear to me just what "Event A" and "Event B" represent. Are these separate events that might or might not happen? Are the events mutually exclusive? Can a participant discuss/share an "Event" that didn't happen? Please edit the question to provide that information, as comments are easy to overlook and can be deleted. $\endgroup$
    – EdM
    Apr 30 at 20:57
  • $\begingroup$ @EdM Hi, thanks for replying! I have edited the question by adding the information you requested. Hope it's now clearer :) $\endgroup$
    – Gloria Ma
    Apr 30 at 22:08

1 Answer 1

2
$\begingroup$

If I understand correctly, Event represents the type of event identified to the investigator (personal versus news story, independent of the particular nature of the event beyond that) and Status is whether an Event was shared by discussing with someone else during a 24-hour period. The outcome is a self-assessment of emotional change in the 24 hours after the initial reporting of the events to the investigator.

You want to model this with fixed effects for Event, Status, and an Event:Status interaction, and a random effect for Participant.

First, there only seems to be a single outcome value for each participant. If that's the case, I don't see what the Participant random effect provides (or if it's even identifiable). That might be different if you had recorded some measure of emotional status prior to and after the study and included both measures for each participant somehow, but that doesn't seem to be what you did if all you have is a change in emotional status.

Second, what you have is simply 4 mutually exclusive sharing situations: neither event shared, only event type A shared, only event type B shared, both event types shared. A way to capture that more directly than your proposed model is to use A-shared and B-shared as two binary predictors, plus an interaction term between them. That will accomplish what you want, as you have a 0/1 no/yes result for each of those individual predictors for all participants.

If "no sharing" is the reference level for each of A-shared and B-shared, the intercept in regression would be the outcome when there was no sharing. The coefficient for A-shared would be the difference from no sharing when A is shared and B isn't; the coefficient for B-shared is similarly the difference from no sharing when B is shared and A isn't. The coefficient for the interaction is the extra difference when both are shared versus what you would predict when each was shared in the absence of the other being shared.

You can then use those coefficient estimates to evaluate your hypotheses. The interaction coefficient directly evaluates whether the effect of A-shared depends on whether there was also B-shared (and vice-versa). The individual coefficients for A-shared and B-shared determine whether either affects outcome (over no sharing) if the other wasn't shared. You can evaluate whether sharing matters at all by testing whether any of the regression coefficients involving sharing differs from 0.

Remember, however: when there is an interaction between two predictors there might no longer be a well-defined "main effect" for either of them. If the effect of one depends on the value of the other, what do you then mean by an individual "main effect"?

One further warning: this study might not be interpretable in the way that you hope, regardless of statistical modeling. Might the choice of sharing either type of event have been driven by an underlying change in emotional status, rather than the reporting changing the emotional status? I don't see a way around that problem.

In response to updated question:

With one outcome for each event type, you can still use a standard linear model, but with a 2-valued multivariate outcome. Each outcome is modeled against the same set of covariates, but the correlations among outcomes can be taken into account. So again you wouldn't have to specify random effects.

If you have measurements of pre-experiment emotional measures for both events, then it would be best to use the actual post-experiment measures as the outcomes, rather than the changes in measures, and to use the pre-experiment measures as predictors. That will still fit into the multivariate regression framework. This page and others inked from there discuss the dangers in using change scores as outcomes. As there are only 2 time points, one at the start and one at the end of the 24-hour experiment, there is no way to distinguish the effect of time from the effect of sharing/not, so there's nothing else you can model.

$\endgroup$
6
  • $\begingroup$ Thanks so much for the answer! However, I think I described the measure of the outcome variable inaccurately (sorry about that): There were actually 2 ratings on perceived emotional change provided by each participant. So, after indicating whether Event A and B were shared, participants rated how much they thought their feelings about EACH event had changed. In this case, I think Participant should be included in the model as a random effect given that there're 2 observations per each participant? $\endgroup$
    – Gloria Ma
    May 1 at 18:22
  • $\begingroup$ Regarding your second point, I imagine that if I re-shape my dataset, instead of two rows per participant (each row containing responses for each event), I will have only one row per participant, which will look like this(?): Participant 1: A-shared (yes/no), B-shared (yes/no), A-emotional-change (rating), B-emotional-change (rating). However, in this case, how can I structure the model given now I have my outcome variable separated in two columns? $\endgroup$
    – Gloria Ma
    May 1 at 18:33
  • 1
    $\begingroup$ @GloriaMa what you describe is a "mutlivariate" (multiple-outcome) regression with only 2 outcomes. That can be handled directly with the lm() function in R, for example, with the outcomes represented as a matrix with 2 columns, one column for each outcome and 1 row per participant. That fits separate models for each of the outcomes, but also recognizes the correlations between the outcomes in a way that takes into account the correlations within each participant. $\endgroup$
    – EdM
    May 1 at 20:09
  • 1
    $\begingroup$ @GloriaMa with a multivariate regression you also wouldn't need to specify random effects for the participants. In that form each participant still has only a single, if 2-valued, outcome, and the within-participant outcome correlations are handled directly. Mixed models have a lot of uses in more complicated situations, but your situation is simple enough for you to use this tried-but-true approach. This document is one useful reference to start. $\endgroup$
    – EdM
    May 1 at 20:13
  • 1
    $\begingroup$ @GloriaMa then don't use change scores; use pre-scores as predictors and post-scores as outcomes. See the addition to the end of the question and the page linked from there. Otherwise the model you propose looks OK. $\endgroup$
    – EdM
    May 1 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.