0
$\begingroup$

I am trying to perform Logistic regression on the sample data set. After its modeling, I tried to check its goodness of fit using the Hosmer Lemeshow test and found the p-value < 0.05, which tells that the model is not a good fit. On, the contrary, when I plotted the ROC Curve for it(another way to check the fitness of model), the curve covered approx 80-90% of the area, which refers the model is suitably fit.

Please find the used dataset named "Urine Analysis Data" in the link: mytestdata

In the Model 1: I cleaned the dataset to remove rows containing NA values, then modeled the dataset and implemented hoslem.test and to draw ROC Curve.

In the Model 2: I tried further cleaning of dataset by removing the independent variables that are not of much significance, then modeled the dataset and implemented hoslem.test and to draw ROC Curve. But, the result dint changed.

Here is the code:

Urinedata <- read.csv("UrineForLogisticRegression.csv", stringsAsFactors = F)
summary(Urinedata)
colSums(is.na(Urinedata))
Urinedata$Calcium.Oxalate <- as.factor(Urinedata$Calcium.Oxalate)
str(Urinedata)

##### Model 1 #####
#Training by Model 1:
Cleaned_data <- Urinedata[complete.cases(Urinedata),]
colSums(is.na(Cleaned_data))
colnames(Cleaned_data)

model1 <- glm(Calcium.Oxalate~gravity+ph+osmo+cond+urea+calc, data = Cleaned_data ,family = binomial)
summary(model1)

#Testing goodness of fit of model using Hoslem Test:
install.packages("ResourceSelection")
library(ResourceSelection)
hoslem.test(Cleaned_data$Calcium.Oxalate, fitted(model1), g=10)
#Conclusion: The p-value is ,much less than 0.05, hence, model not a good fit.

# Predicting same dataset values using Model1 now:
install.packages("ROCR")
library(ROCR)
predictedCalciumOxalate <- predict(model1, Cleaned_data, type = "response")
test_output <- cbind(testdata,predictedCalciumOxalate)

#Plotting ROC graph for data, if area more, then good fit:
length(fitted(model2))==length(more_clean_data$Calcium.Oxalate)
preds <- prediction(as.numeric(predictedCalciumOxalate), as.numeric(Cleaned_data$Calcium.Oxalate))
perf <- performance(preds,"tpr","fpr")
plot(perf)



#####I also tried Model 2#####
#Model 2 is determined by doing more cleansing on data

more_clean_data <- Cleaned_data[,c(1,6,7)]
colSums(is.na(more_clean_data))
colnames(more_clean_data)
model2 <- glm(Calcium.Oxalate~urea*calc, data = more_clean_data, family = "binomial")
summary(model2)
hoslem.test(more_clean_data$Calcium.Oxalate, fitted(model2))
#Hoslem test giving same result as model1 , thus dint proceeded with Model2 further.

I am expecting that if Hosmer lemeshow test is telling the model is not a good fit, then ROC curve should also have reflected the same. I am new to R. Please correct if am wrong somewhere in the above statement conceptually.

$\endgroup$

migrated from stackoverflow.com Dec 22 '18 at 16:15

This question came from our site for professional and enthusiast programmers.

  • 2
    $\begingroup$ Hosmer–Lemeshow test provides information about calibration while ROC curve is more about discrimination. Additionally, cut-off values depend on the context of the study. i.e. an AUC of 80-90% is not necessarily an indicator of good fit. The Cross-Validated community would better address and explain this question. $\endgroup$ – Ozan147 Dec 22 '18 at 5:36
  • $\begingroup$ See stats.stackexchange.com/questions/164204/… $\endgroup$ – kjetil b halvorsen Dec 22 '18 at 22:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.