I am trying to perform Logistic regression on the sample data set. After its modeling, I tried to check its goodness of fit using the Hosmer Lemeshow test and found the p-value < 0.05, which tells that the model is not a good fit. On, the contrary, when I plotted the ROC Curve for it(another way to check the fitness of model), the curve covered approx 80-90% of the area, which refers the model is suitably fit.
Please find the used dataset named "Urine Analysis Data" in the link: mytestdata
In the Model 1: I cleaned the dataset to remove rows containing NA values, then modeled the dataset and implemented hoslem.test and to draw ROC Curve.
In the Model 2: I tried further cleaning of dataset by removing the independent variables that are not of much significance, then modeled the dataset and implemented hoslem.test and to draw ROC Curve. But, the result dint changed.
Here is the code:
Urinedata <- read.csv("UrineForLogisticRegression.csv", stringsAsFactors = F) summary(Urinedata) colSums(is.na(Urinedata)) Urinedata$Calcium.Oxalate <- as.factor(Urinedata$Calcium.Oxalate) str(Urinedata) ##### Model 1 ##### #Training by Model 1: Cleaned_data <- Urinedata[complete.cases(Urinedata),] colSums(is.na(Cleaned_data)) colnames(Cleaned_data) model1 <- glm(Calcium.Oxalate~gravity+ph+osmo+cond+urea+calc, data = Cleaned_data ,family = binomial) summary(model1) #Testing goodness of fit of model using Hoslem Test: install.packages("ResourceSelection") library(ResourceSelection) hoslem.test(Cleaned_data$Calcium.Oxalate, fitted(model1), g=10) #Conclusion: The p-value is ,much less than 0.05, hence, model not a good fit. # Predicting same dataset values using Model1 now: install.packages("ROCR") library(ROCR) predictedCalciumOxalate <- predict(model1, Cleaned_data, type = "response") test_output <- cbind(testdata,predictedCalciumOxalate) #Plotting ROC graph for data, if area more, then good fit: length(fitted(model2))==length(more_clean_data$Calcium.Oxalate) preds <- prediction(as.numeric(predictedCalciumOxalate), as.numeric(Cleaned_data$Calcium.Oxalate)) perf <- performance(preds,"tpr","fpr") plot(perf) #####I also tried Model 2##### #Model 2 is determined by doing more cleansing on data more_clean_data <- Cleaned_data[,c(1,6,7)] colSums(is.na(more_clean_data)) colnames(more_clean_data) model2 <- glm(Calcium.Oxalate~urea*calc, data = more_clean_data, family = "binomial") summary(model2) hoslem.test(more_clean_data$Calcium.Oxalate, fitted(model2)) #Hoslem test giving same result as model1 , thus dint proceeded with Model2 further.
I am expecting that if Hosmer lemeshow test is telling the model is not a good fit, then ROC curve should also have reflected the same. I am new to R. Please correct if am wrong somewhere in the above statement conceptually.