1
$\begingroup$

I am trying to compare two models and check which is the best fit of our data. The R script is below:

# Models
fit1<-glm(bp_target~gender+age, family=binomial(link = "logit"))
fit2<-glm(bp_target~gender+age+tc+hdl+factor(smoking)+glucose, family=binomial(link="logit"))
#Goodness of fit
anova(fit1, fit2, test="Chisq")

And we got the warning:

Error in anova.glmlist(c(list(object), dotargs), dispersion = dispersion,  : 

models were not all fitted to the same size of dataset

I do understand anova() can only be used to check goodness of fit when two models have equal dataset. However, in my case, fit2 has more missing values than fit1.

How could I compare these two models and decide which is better?

$\endgroup$
-1
$\begingroup$

Because the second model has more variables, you're more likely to get missing values. So I think there are two/three things you could do:

  1. If there aren't that many missing values, you can look at just removing them and be done with them. But then when you are using your model to make a prediction, you can't predict on those customers who have missing data.

  2. You can look into the amelia package in R which gives a solution to replace the missing value by using multiple imputation. This again might pose a problem when predicting for people with missing values.

  3. Both the options above take away some information value from the data, because we no longer have the effect of 'missing' data in our model. In fact, this already is happening with your models since logistic regression just removes any row that has a NA value in one of its factors. So then, instead of trying to get rid of these missing values, you could replace them with a certain value such as $-9999$ if it is a numerical or simply just 'missing' if categorical. In terms of categorical, it will just become a new level so it works out great. But when it comes to numerical, you may have to look into binning your variable so that all missing values are in the same bin. This goes into WoE and information value, which I think this link would be useful for: http://ucanalytics.com/blogs/information-value-and-weight-of-evidencebanking-case/

In the end you have to make the decision of if you're ok with making certain trade-offs, for example maybe your current data set has missing values but for your future data that you'll collect you can make sure you won't have any missing values so you can then go ahead and remove missing values all together. It all depends on your situation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.