1
$\begingroup$

Suppose I fit a generalized mixed logistic model such like that:

set.seed(2014)
require(lme4)
df<-data.frame(id=rep(1:5, c(8,10,12,14,15)), 
               out=c(rbinom(8,1,0.1), rbinom(10,1,0.3),rbinom(12,1,0.1),rbinom(14,1,0.05),rbinom(15,1,0.1)),
               age=rnorm(59,50,10),
               gender=rbinom(59,1,0.5))
fit<-glmer(out~age+gender+(1|id),data=df,binomial)
df$predicted<-predict(fit,type="response")
df$pred.binary<-with(df,ifelse(predicted>=0.5,1,0))
apply(df[,c(2,6)],2,sum)
        out pred.binary 
          9           0 

What's the best way to predict a binary outcomes by accounting the random effects and with an optimized threshold based on the original data set?

$\endgroup$
3
  • 1
    $\begingroup$ start with predict (i.e. ?predict.merMod) with type="response" to get probabilities, and then tell us more about your definition of an optimal threshold ... $\endgroup$
    – Ben Bolker
    Mar 25, 2014 at 13:02
  • $\begingroup$ From my updated demo, you may see, if I use 0.5 as a critical value, the predicted cases will be zero comparing to the 9 cases in the observed out. I got same situation with my real dataset with 25,00 rows and predicted zero case too. Thus I'd think it's necessary to find an optimal threshold. $\endgroup$
    – David Z
    Mar 25, 2014 at 17:01
  • $\begingroup$ to find an optimal threshold you need to know your loss function.... $\endgroup$
    – charles
    Apr 25, 2014 at 21:38

2 Answers 2

1
$\begingroup$

Well, i believe a common practice is to use a threshold of p=0.5 for the predicted response which is in ]0 1[. Not sure how you can optimize this threshold? (I assume gender and age are between-subjects effects)

EDIT: I see. I would say the problem here is not in the threshold, but in the data/model: the overall chance of out==1 is low and the predictors of the model don't tell you much about the outcome. Hence, the prediction is out==0 for all data because, overall, this is the most probable case.

$\endgroup$
2
  • $\begingroup$ See my updated demo and the comment above. $\endgroup$
    – David Z
    Mar 25, 2014 at 17:02
  • 1
    $\begingroup$ might be worth adding that threshold of 0.5 implies a loss function of 1:1 $\endgroup$
    – charles
    Apr 25, 2014 at 21:40
0
$\begingroup$

Two options for predicting binary outcome:

1. yhat<-rbinom(nrow(df), 1, predicted)
2. u<-runif(nrow(df))
    yhat<-ifelse(predicted>=u, 1, 0)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.