Larger p-values but less misclassification error in Logistic Regression I was doing logistic regression in R on 'Smarket' data set available in the ISLR library.
Since correlation between variables were less, I used all variables in my model and I was getting the following result

Here p -values are greater then 0.05 for all the variables.
Then I checked the misclassification error and the error was almost zero.

I have tried removing some variables. But still the p values are greater than 0.05 and the misclassification error rates are high for those models.
Can I use this model ??
Is p-value insignificant in logistic regression?
 A: Here's your problem, and it comes down to basic arithmetic: you are modeling Direction, which is a binary variable (Up or Down), and your independent variables define it perfectly. 
Direction=Up where Today>Lag1
Direction=Down where Today

The arithmetic in your model shows you the mistake:
Direction=Today+Lag1+....
Essentially, because the definition of Direction is built in to the formula, you have for many cases a tautology: price (I assume that's what it is) increases because it increases.
The p-values are high because I suspect (though I don't know for sure) that glm (if that's what you're using, you don't say) is excluding the perfect predictions from the results. According to my output:
glm(Direction~Lag1+Lag2+Lag3+Lag4+Lag5+Volume+Today,data=Smarket,family="binomial")
Warning messages:
1: glm.fit: algorithm did not converge 
2: glm.fit: fitted probabilities numerically 0 or 1 occurred 

Even if you ignore those pretty important warnings, your table should also give you pause. There are 1250 rows in Smarket; your 2x2 table only has 252.
