2
$\begingroup$

I have a logistic mixed model where a random effects term I am thinking about including is totally arbitrarily related (in a non-informative way) to the response variable. I would not include it as a predictor or a fixed effects term, because it would do a great job of predicting the response variable and not tell me anything about the system. However, I want to include it as random effects term because it may help account for variability. It's like using zip code to predict epidemic location across cities when you already know the epidemic only occurs in certain cities, but zip code helps control for a number of factors influencing epidemic. When I include this term as a random intercept term (1|x in lmer in R) my prediction accuracy jumps 15% and the fixed terms all lose significance. Is the correlation between the response and random terms helping the model perform better?

Made up example code (pretend district is subset of zip code):

zip<-c(54403,54403,54403,80404,80404,80405,93513,93513,93514,30411,30412,
30413,60414,60415,601415)

city_has_epi<-c(yes,yes,no,no,no,no,yes,yes,yes,no,no,yes)

avg_income<-c(50000,40000,35000,80000,90000,400000,50000,40000,35000,
80000,90000,400000)

hospitals_no<-c(23,46,66,29,44,54,23,46,66,29,44,54)

district<-c(1,2,3,4,5,6,7,8,9,10,11,12)

model<-glmer(city_has_epi~ avg_income + hospitals_no +(1|zip/district), family=binomial(link="logit"), data=mtr, control=glmerControl(optimizer="bobyqa"))
$\endgroup$
  • 1
    $\begingroup$ it would be helpful to know how you are quantifying prediction accuracy- an R2 metric, leaving data out, or something else? Also- any test data and code you could provide in the body of your question would be helpful for providing a thorough response! $\endgroup$ – colin Jun 1 '16 at 14:06
  • $\begingroup$ Thank you, @colin. I'm witholding 3/4 of the data to test prediction accuracy. Since it's a logistic model, I'm not using a r2 metric. $\endgroup$ – Mina Jun 1 '16 at 15:21
  • $\begingroup$ The example here is different than my actual research question (described here: stats.stackexchange.com/questions/214373/…). A colleague of mine suggested I reframe the question because it might be confusing to non-natural resource types. $\endgroup$ – Mina Jun 1 '16 at 15:23
1
+50
$\begingroup$

The problem here is not that your random effect and your response variable are somewhat correlated, its that they are perfectly correlated. The model estimates a baseline level of the dependent variable for each random effect in the model. So, here the levels of your random effect can explain ALL of the variation in your response variable.

Generally, when using something like zip code as a random effect in the context of disease, people are analyzing occurrence rates, given some baseline rate within that community modeled by the random effect, as a function of predictors that may influence that occurrence rate. Or, you might analyze the effect of income and water source on lead levels in blood, while using zip code of those people as a random effect, to account for the fact that there are other environmental factors that amy be geographically unique but unobserved that drive variation in baseline lead levels in blood.

In this example, given these data, the incorporation of zip code as a random effect is inappropriate, as it cannot give you baseline information on this binary outcome. Your dependent variable and random effect are perfectly correlated, and this is why you are having trouble.

$\endgroup$
  • $\begingroup$ Thank you, @colin. My example wasn't great, I realize now. I edited it so the data doesn't totally correlate, which is the case with my actual situation. $\endgroup$ – Mina Jun 1 '16 at 19:07
  • $\begingroup$ I am really modeling the vegetative differences between active bird lek areas and inactive areas. Active lek areas are spatially coincident with grazing allotment on national grasslands, which is partially due to their large size and that there are not many distinct areas. Most plots that fall within a given allotment within an active lek area, also are in the active lek area and vice versa. I want to include allotment as a random effect because there may be grazing differences among allotments. $\endgroup$ – Mina Jun 1 '16 at 19:14
  • $\begingroup$ I assume that even if my example random effects data were not perfectly correlated to the response, given your answer to my question, it would still be problematic to include them as a random effect @colin? $\endgroup$ – Mina Jun 1 '16 at 19:17
  • 1
    $\begingroup$ @Mina hard to catch everything that you mean from your comment, but it sounds like allotment (and therefore grazing treatment), and lek areas are highly correlated. If this is the case you may not have enough power to separate these statistically, which is why you see few other effects. I would suggest doing a power analysis, where there is a large effect of lek, independent of allotment. Generate random data, given your sample numbers and how they are distributed, and determine if its even possible to detect an effect with your design. $\endgroup$ – colin Jun 1 '16 at 19:18
  • 1
    $\begingroup$ @Mina if you have a continuous response variable (like vegetation), there is a better shot. But if they are too confounded, then you need to have enough sampling effort to detect lek effects that are 90% confounded with 'allotment' effects. $\endgroup$ – colin Jun 1 '16 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.