2
$\begingroup$

I am creating a multilevel model to measure change over time in risk behaviors over 25 months. The risk behaviors are calculated as how many behaviors from a list of 22 that people report doing or not doing in any given month. However, because of logic statements, participants may or may not be asked about 2 of the 22 behaviors. Therefore, for some people, the max number of behaviors is 20, for others the max is 21, and for the rest the max is 22.

I have been treating this as a proportion, and dividing the total number of risk behaviors by a variable for number of items that the person was asked. I also considered using nofitems as a covariate.

But could I use a random effect to account for that variation? How would that change the interpretation of my model?

Is one or more of the following more appropriate than others for handling the difference in number of items asked?

Covariate:

lmer(Outcome ~ time+nofitems +(1|id),data)

Random Effect

lmer(Outcome ~ time +(nofitems|id),data)

Divide by Total Possible

glmer(Outcome/nofitems ~ time +(1|id),data,family='binomial,weights=nofitems)'

Divide by Total Possible AND Use Random Effect

glmer(Outcome/nofitems ~ time+ (nofitems|id),data,family='binomial,weights=nofitems)'

Improve this question
$\endgroup$
1
  • $\begingroup$ If the logic excludes impossible behaviors, you might want to look at count models instead of using proportion, eg if risky behaviors are rare, a poisson or negative binomial $\endgroup$
    – 5ayat
    Commented Oct 5, 2017 at 14:14

1 Answer 1

Reset to default
2
$\begingroup$

I don't think including the total number of items as a random effect makes sense. The approach where you divide by the total number is equivalent to estimating a binomial model. So:

glmer(Outcome/nofitems ~ time +(1|id),data,family='binomial',weights=nofitems)

is equivalent to:

glmer(cbind(Outcome,nofitems - Outcome) ~ time +(1|id),data,family='binomial')

This is the model talked about in Britt et al. (2017), The analysis of bounded count data in criminology. In reference to the comment on your post, the reason count models are not appropriate is that you have an upper bound due to the nature of the instrument. If you only have 22 deviant questions you can never have a person with 23 or more counts of deviance within any time frame. Poisson models will often have a significant amount of predicted mass above the natural cut-off, so are from the start not a very good fit to the data. The binomial model approach implicitly takes into account the differing number of items for folks, so no need to additionally include it as a covariate.

Now, this isn't my area of expertise, but I'm surprised folks think the binomial models are reasonable. The binomial model produces equivalent results to predicting the individual delinquency items, but forcing them to have the same probability of occurring. For risky behaviors, every dataset I've seen has some that are more common (e.g. smoking cigarrettes, drinking alcohol) versus behaviors that are much rarer (e.g. using heroin). The binomial model forces these to all have the same underlying predicted probability per your person covariates in the sample.

It seems to me a better approach is to let those different risk factors have a varying term to account for the different prevalence of the items. (This is very similar to item response theory -- some questions are harder and some are easier, see Osgood et al. (2002), Analyzing multiple-item measures of crime and deviance: Item response theory scaling, for another deviant behavior example.)

So here is some prototype R code, pretend your original deviance measures are in variables D1, D2, D3, etc. Here I am suggesting you reshape your data from wide to long format:

dVars <- c("D1","D2","D3")
long_data <- reshape(data, varying = dVars, v.names = "DelinQ", timevar = "OutcomeI", times = dVars, direction = "long")

Then the two models below are equivalent.

glmer(cbind(Outcome,nofitems - Outcome) ~ time +(1|id),data,family='binomial') #Model with the wide data
glmer(OutcomeI ~ time + (1|id),long_data,family='binomial') #Model with the long data

But I think it probably makes sense to let the individual items have their own baseline rates of probability of occurring. You can either do this via random effects or fixed effects. (With only 22 items, which is alot, it is still hard to estimate the random effect distribution.)

glmer(Delin ~ time + DelinQ + (1|id),long_data,family='binomial') #Fixed effect for question
glmer(Delin ~ time + (1|DelinQ) + (1|id),long_data,family='binomial') #Random effect for question

This approach generalizes to items missing for some folks, same as the aggregated binomial approach.

For those who use Stata I've generated a simulation for this exact situation and illustrated the different models discussed here.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks, @Andy! I wonder if a simple proportion is acceptable b/c of why they didn't get the questions. Ex: It was only a risk behavior to take drug X if you weren't being prescribed drug X. People who have prescription might be different! A covariate for this might be best. I agree about binomial models. Assumptions are too strict for the real data I've seen. I've been using a beta-binomial model with glmmADMB to estimate that probability term with a beta-distributed random effect (a common model for overdispersion). Do you think that's as good of an approach as the ones suggested here? $\endgroup$
    – mrjaws
    Commented Oct 5, 2017 at 20:15
  • $\begingroup$ Also, I voted for but didn't select your thorough, informative answer because it mostly answers a question I didn't ask: one about the feasibility of using binomial models for these data & ways to handle the differing probabilities of such behaviors. Useful, though. But perhaps my question's phrasing left something to be desired. How bout this: Since subgroup logic was used (e.g. "Do you have prescription for drug X?" Yes=21 Q's, No=22 Q's), the # of questions a person received might be correlated with their outcome. Would a random effect capture this, or would a covariate be better? Why? $\endgroup$
    – mrjaws
    Commented Oct 5, 2017 at 20:38
  • 1
    $\begingroup$ The beta-binomial model has the same critique that the binomial one does. One way I've seen skip logic addressed is to consider it a type of missing data. IIRC, if you include a dummy variable covariate that measures why the skip was used (here if they were prescribed medication) then the missing items become MCAR. The individual logit approach I suggested then is not biased by dropping those observations -- and by extension the binomial model is fine as well. $\endgroup$
    – Andy W
    Commented Oct 6, 2017 at 12:13
  • $\begingroup$ Hi @Andy. Thanks again! So, to be clear, I want to restate what you said. 1. The beta-binomial model has the same critique because, although it estimates the probability as another parameter and thus allows for some uncertainty, it still doesn't allow each behavior to have its own probability and is inherently flawed in that way. Correct? Secondly, including a logic covariate should handle the bias introduced when skip logic is correlated with the outcome. In other words, "Holding cov constant, the main effect of x on (risk behaviors / num. of behaviors) is expected to be [beta]." Right? $\endgroup$
    – mrjaws
    Commented Oct 6, 2017 at 13:04
  • $\begingroup$ Yes you got it for both. $\endgroup$
    – Andy W
    Commented Oct 6, 2017 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.