So I'm working with logistic regression models in R. Though I'm still new to statistics I feel like I got a bit of an understanding for regression models by now, but there's still something that bothers me:
Looking at the linked picture, you see the summary R prints for an example model I created. The model is trying to predict, if an email in the dataset will be refound or not (binary variable
isRefound) and the dataset contains two variables closely related to
isRefound , namely
next7days - these are also binary and tell if a mail will be clicked in the next 24hrs / next 7 days from the current point in the logs.
The high p-value should indicate, that the impact this variable has on the model prediction is pretty random, isn't it? Based on this I don't understand why the precision of the models predictions drops below 10% when these two variables are left out of the calculation formula. If these variables show such a low significance, why does removing them from the model have such a big impact?
Best regards and thanks in advance, Rickyfox
First I removed only next24, which should yield a low impact because it's coef is pretty small. As expected, little changed - not gonna upload a pic for that.
Removing next7days tho had a big impact on the model: AIC 200k up, precision down to 16% and recall down to 73%