1
$\begingroup$

I have run so many models and am so, so confused.

I've attached an image of sample data that matches my structure. I have a single score per conversation (score is my outcome variable). For each conversation, I have multiple measurements of predictors of interest (illustrated predictor_1). My measurements for predictor_1 are taken on a fine grained scale, such that I have 200 rows for each conversation_id. Subjects will have multiple conversations (each conversation_id is unique) and session_n up to 20. I am interested in the effects of my predictor_ns, session_n, and gender. I know I want a need a random intercept for client, but I feel like I need some way to capture the fact that I have repeated measurements for each conversation_id. Is it okay that some of my predictors are captured on a more fine-grained scale than my outcome variable?

My initial model:

lm.mymod  <- lmer(score ~ (1|subject_ID) + gender + session_n + predictor_1 + predictor_2, data = data)

I don't think this is right - the effects of gender and session_n are super strong because it's treating it as if I more measurements for those than I really do.

I feel like the easiest thing to do would be to average my predictor_ns across conversation_id - but is there a way I can represent this in a nested way?

This seems sensible but fails to converge, I think b/c I only have one value for score for each conversation_id:

lm.mymod  <- lmer(score ~ (1|subject_ID / conversation_id) + gender + session_n + predictor_1 + predictor_2, data = data)

I'm interested in the fixed effect of session_n (I expect it to increase with my score) so I don't want to do:

lm.mymod  <- lmer(score ~ (1|subject_ID) + (1|session_n) + gender + predictor_1 + predictor_2, data = data)

I would really appreciate any help or guidance. I'm at the point where I might just average across conversation_id.

Sample data structure

$\endgroup$
2
  • 1
    $\begingroup$ Within the multilevel model framework it's not possible to use predictors with more observations per cluster than the dependent has. The dependent variable dictates the model you can use. The approach suggested by Ggjj11 seems promising though. $\endgroup$
    – Sointu
    Apr 16 at 6:24
  • 1
    $\begingroup$ To clarify: you can of course use the predictors, but not all the fine-grained info they carry. You need to average them e.g. by subject and session. $\endgroup$
    – Sointu
    Apr 16 at 6:58

1 Answer 1

1
$\begingroup$

Very interesting. Note that you also have nesting due to a session_n variable (in a way a time variable)

It reminds me of the n:1 models and 1:1 models in time series regression (be aware quite some transfer needed for getting my answer)

In short, my take:

  • the n:1 model takes all n separate time series (groups) to build a single model
  • the 1:1 model constructs n models for each time series separately (good if the time series/groups are independent of each other)

But better read here for the discussion in time-series https://learn.microsoft.com/en-us/azure/machine-learning/concept-automl-forecasting-methods?view=azureml-api-2

Now you have even more nesting and you might go with building hirachical regression models or you can apply dimensionality reduction in a some way or the other.

It really depends on what you want to achieve. The way of grouping definitely also depends on what you consider information leakage and what is fine.

Maybe others can give you more direct insight.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.