R logistic regression [closed]

Question

I was using the Smarket data in the ISLR package for logistic regression (in R) and find the error below. If I use the following command, I get the error (warning):

glm.fit1 = glm(Direction~. - Direction, data=Smarket, family="binomial") 
Warning messages: 1: glm.fit: algorithm did not converge 
                  2: glm.fit: fitted probabilities numerically 0 or 1 occurred

However if I run the following command, I do not see any error:

glm.fit = glm(Direction∼Lag1+Lag2+Lag3+Lag4+Lag5+Volume, data=Smarket, 
              family=binomial)

Although both the commands looks same to me (excluding the direction), the outputs are different. So, my questions are

1. Why am I getting the warning if both the commands are same?
2. Why is the output also different?

Answer 1

It does seem off topic, but with respect to why you are getting the warning message, the warning is indicating that linear separation is present in the model. This most often occurs when there are several covariates $X_1, X_2, ... , X_k$ and there is a linear function:

$L = \beta_1X_1 + \beta_2X_2 + ... + \beta_kX_k$

for which (with the data as observed) there is a value $L_0$ such that

$Y = 0$ for all the data such that $L < L_0$ $Y = 1$ for all the data such that $L > L_0$

So essentially a perfect fit is possible within the parametrisation of the model.

Are you sure the exact same model is being fit in both cases? It would be best to double check that if you are getting different outputs.

Answer 2

From the documentation for ?Smarket, it doesn't appear to me that your two function calls are the same. The variables included are reported to be:

A data frame with 1250 observations on the following 9 variables.

Year The year that the observation was recorded
Lag1 Percentage return for previous day
Lag2 Percentage return for 2 days previous
Lag3 Percentage return for 3 days previous
Lag4 Percentage return for 4 days previous
Lag5 Percentage return for 5 days previous
Volume Volume of shares traded (number of daily shares traded in billions)
Today Percentage return for today
Direction A factor with levels Down and Up indicating whether the market had a positive or negative return on a given day

That is, your first function call would include today, but your second call does not. In addition, you do not need to subtract the response variable from the RHS when using ~.. That is, you can just use:

glm(Direction~., data=Smarket, family="binomial")