I was trying to help a student and was foxed by this Logistic Regression problem and seek your explanation.
Here's some economic data. All we have to do is create a model to predict whether the country falls in debt. The response variable is "Dependent.variable"
I analyzed using 'R' and the glm function and what I found was:
- All the variables showed a 'p-value' of 1, indicating none of the variables are significant.
- I plotted each variable versus the dependent - and sure enough - the variable is not co-related to the response. The feature values are scattered all over the place
- And yet - the model fits perfectly! On a test set - it predicted all cases perfectly
- The ROC curve is a rock solid square
- I then took all the "coefficients estimates" and built the equation and computed probabilities using the expanded form of e/(1+e) - and it does give near perfect probabilities
How is this possible? What am I missing here!
Edit: Here is the model and the output
Call: glm(formula = Dependent.variable ~ ., family = binomial, data = dfTrain) Deviance Residuals: Min 1Q Median 3Q Max -4.107e-05 -2.100e-08 -2.100e-08 2.100e-08 4.278e-05 Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) -3.238e+01 6.458e+05 0.000 1.000 Inflation.CPI. 1.931e+00 4.966e+03 0.000 1.000 Debt.GDP 2.590e-01 1.830e+03 0.000 1.000 OfficialForexReserves.US.Bil.. -2.575e+00 2.538e+03 -0.001 0.999 GrossInvestment.GDP 7.396e+00 6.158e+04 0.000 1.000 Gov.Revenue.GDP -5.396e+00 3.016e+04 0.000 1.000 Net.FDI.GDP 8.927e-01 1.135e+04 0.000 1.000 (Dispersion parameter for binomial family taken to be 1) Null deviance: 6.4924e+01 on 50 degrees of freedom Residual deviance: 6.0091e-09 on 44 degrees of freedom AIC: 14 Number of Fisher Scoring iterations: 25