# Best method to predict binary outcome with multiple records per subject

I am interested in building a model to predict the binary outcome, retention (1 - retained; 0 - not retained) with various potential predictor variables (either continuous or categorical).

With that being said, I have a dataset containing multiple records (magazine subscriptions) for some subjects. For example, I have 4 records at the magazine subscription level for Joe Smith (3 of which he is retained, while the other record he is not retained) and 7 magazine subscription records for John Doe (4 of which he is retained, while the other 3 records he is not retained).

My initial thoughts were to use logistic regression using a single randomly selected magazine subscription for each subject. For example, randomly selecting 1 of the 4 magazine subscriptions for Joe Smith and 1 of the 7 magazine subscriptions for John Doe. Obviously, I would be losing a great deal of my data, which makes me think that there has to be a better way of modeling this data.

What method would you be best to predict retention with data such as this?

Thank you!

• It would maybe help if you briefly describe how the multiple records for each subject were created. Are these repated measures of the the same 'concept'? Are these measures ordered or is their order unsystematic? Is there a continuous time variable underlying the distance of measurement moments or can they be considered discrete time intervals? All of this information may help choosing a correct model for the outcome variable. Jul 20 '14 at 9:20
• The multiple record per unit of observation leads me to think that perhaps you are interested in the rate of retention, provided you also captured some measures of time. You can then make use of the vast literature on repeated events survival analysis. Jul 21 '14 at 13:54
• To address @tomka's comment, I have included more specifics regarding my post. Does this provide anyone with additional information to better answer the question at hand? Jul 21 '14 at 19:33
• @ThomasSpeidel, we are not interested in rate of retention, but I appreciate your observation. We are interested in whether or not these magazine subscriptions are retained (re-subscription) or not retained (no more subscriptions). Jul 22 '14 at 13:45
• @MattReichenbach ok. But just to clarify, if retention is the event and you have the time of origin then you can study if predictors of interest make people unsubscribe at a faster rate. It's another way of framing the problem, though I don't know enough about your problem. Jul 22 '14 at 13:59

A large part of the problem seems to be that the data structure is not amenable to analysis. Is there any reason you can't collapse the data into one record per person-year (e.g you're using more than just the variables from the subrecords)? The result may have a heck of a lot of indicator variables, but it would make the analysis a lot easier.

Due to the different requirements of the data at various stages at my company, I've had to collapse data in that manner countless times.

• So are you saying, if I have had a dataset with 2 records for some individuals and 1 record for the rest (to keep it simple), create 2 sets of variables (for example, customer age), one that pertains to the first record's age (for example, age1) and one that pertains to the second record (age2, when available)? You seem to have a grasp for my challenge, and I really appreciate your insight. Thank you! Jul 25 '14 at 19:04
• Sigh. Apparently I'm not the first guy to have this blinding insight. Database guys even have a name for the process: denormalization ;) Jul 25 '14 at 19:27
• Good to know the DBA buzzword, this may prove helpful in itself! Thank you! Jul 25 '14 at 19:56
• My main driver for creating this question was to figure out a way to model this data, as is, as doing binary logistic regression with this data just doesn't seem like the best approach, but apparently there really isn't a good way to tackle this problem as is...denormalization, here we come! Jul 25 '14 at 19:57

I'm not an expert on this but to me it looks like you should use a logistic mixed effects regression model (i.e., with a logit link). This lets you declare a random intercept (and slope if required) for every subject and thus account for the dependence of the observations. This way you can use all the records and preserve all the data and variance. In R, lmer in the lme4 package can do this by choosing the "binomial" family. The model would look roughly like this (if using only a random intercept):

library(lme4)
model <- lmer(retention <- predictor1 + predictor2 + (1|subjectid), family=binomial)


I hope this is what you were looking for.

Edit: Because I'm currently working on a similar problem, I'll have to update this with some current insights. It is probably not a good idea to just modell a random intercept by (+ 1|subject) but also to assume that your one or all of your predictor variables may have a different effect for every subject by putting (1+predictor1+predictor2|subjectid), thus additionally modelling a random slope.

• Is the key word for this "mixed effect"? Am I right in assuming that this can be applied to ordinary regression (continuous outcome)? Jul 18 '14 at 9:46
• @Zhubarb Yeah, if you want to use it on a continuous outcome, you just drop the "family=binomial" part. Jul 18 '14 at 10:03
• @SpookyFM The second half of my answer here addresses some non-obvious issues and proposes a few solutions with using mixed models for risk prediction. Jul 18 '14 at 19:01

An almost optimal approach can be to fit ordinary binary regression model, possibly with updated (time-dependent) covariates, to all the records, and to just not trust the standard errors that result. You can use the cluster sandwich covariance matrix estimator or cluster bootstrap to get "honest" standard errors. I say "almost" because a mixed effects or other full likelihood approach that takes the particular form of intra-subject correlation into account can result in more efficient estimates of $\beta$ coefficients. But this "working independence model" approach is often not far from optimal, as long as you compute the variance-covariance matrix so as to take into account redundancies across records within subject. In the R rms package the pertinent functions are lrm, robcov, bootcov.

• Is the sandwich estimator the same thing as the GEE estimator? Jul 18 '14 at 19:30
• GEE is a method of fitting models as I described. We usually don't use the phrase "GEE estimator" for describing the variance-covariance matrix estimator but focus on the name of a particular estimator that can be used with or without the context of GEE: the sandwich estimator. The cluster bootstrap is a competitor of that. Jul 18 '14 at 20:54