I was using the Smarket data in the ISLR package for logistic regression (in R) and find the error below. If I use the following command, I get the error (warning):

glm.fit1 = glm(Direction~. - Direction, data=Smarket, family="binomial") 
Warning messages: 1: glm.fit: algorithm did not converge 
                  2: glm.fit: fitted probabilities numerically 0 or 1 occurred

However if I run the following command, I do not see any error:

glm.fit = glm(Direction∼Lag1+Lag2+Lag3+Lag4+Lag5+Volume, data=Smarket, 

Although both the commands looks same to me (excluding the direction), the outputs are different. So, my questions are

  1. Why am I getting the warning if both the commands are same?
  2. Why is the output also different?

From the documentation for ?Smarket, it doesn't appear to me that your two function calls are the same. The variables included are reported to be:

A data frame with 1250 observations on the following 9 variables.

Year The year that the observation was recorded
Lag1 Percentage return for previous day
Lag2 Percentage return for 2 days previous
Lag3 Percentage return for 3 days previous
Lag4 Percentage return for 4 days previous
Lag5 Percentage return for 5 days previous
Volume Volume of shares traded (number of daily shares traded in billions)
Today Percentage return for today
Direction A factor with levels Down and Up indicating whether the market had a positive or negative return on a given day

That is, your first function call would include today, but your second call does not. In addition, you do not need to subtract the response variable from the RHS when using ~.. That is, you can just use:

glm(Direction~., data=Smarket, family="binomial")
| cite | improve this answer | |
  • 2
    $\begingroup$ in other words, of course if the OP wants to predict whether the return was positive or negative on a given day, i.e., Direction, given the value of the return on that same day Today, he/she gets a linearly separable problem and issues arise. If instead he/she leaves Today out from the list of predictors, everything makes sense and glm runs smoothly. Of course predictions from this model are garbage since today's returns are (nearly) unpredictable given returns in the preceding days, but at least the model can be fit and used. $\endgroup$ – DeltaIV May 15 '17 at 16:10

It does seem off topic, but with respect to why you are getting the warning message, the warning is indicating that linear separation is present in the model. This most often occurs when there are several covariates $X_1, X_2, ... , X_k$ and there is a linear function:

$L = \beta_1X_1 + \beta_2X_2 + ... + \beta_kX_k$

for which (with the data as observed) there is a value $L_0$ such that

$Y = 0$ for all the data such that $L < L_0$ $Y = 1$ for all the data such that $L > L_0$

So essentially a perfect fit is possible within the parametrisation of the model.

Are you sure the exact same model is being fit in both cases? It would be best to double check that if you are getting different outputs.

| cite | improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.