0
$\begingroup$

let m be my matrix of data

      x_i y_i
 [1,] 0.0   0
 [2,] 0.0   0
 [3,] 0.0   0
 [4,] 0.0   0
 [5,] 0.1   0
 [6,] 0.2   0
 [7,] 0.3   0
 [8,] 0.4   0
 [9,] 0.5   0
[10,] 0.6   0
[11,] 0.0   1
[12,] 0.0   1
[13,] 0.0   1
[14,] 0.9   1
[15,] 1.0   1

My aim is to study the logistic regression y~x, where the covariate x has observations m[,1] and similarly for y. Please note that we have no complete separation in the data but the "anomalous" entries in rows m[11,], m[12,] and m[13,] all correspond to observations with x_i=0.

I expect glm to diverge as the likelihood function reaches no maximum in the ray $k\beta$, for $k\rightarrow \infty$ and $\beta=(-0.7,1)$.

Using glm with 1 iteration I get the output

  Coefficients:
            Estimate Std. Error z value Pr(>|z|)
(Intercept)  -1.2552     0.7648  -1.641    0.101
x             1.6671     1.7961   0.928    0.353

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 19.095  on 14  degrees of freedom
Residual deviance: 18.275  on 13  degrees of freedom
AIC: 22.275

Number of Fisher Scoring iterations: 1

with an error message (the algorithm does not converge). Moreover, with the default number of iterations (=25) the output is

Coefficients:
            Estimate Std. Error z value Pr(>|z|)
(Intercept)  -1.1257     0.7552  -1.491    0.136
x             1.4990     1.6486   0.909    0.363

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 19.095  on 14  degrees of freedom
Residual deviance: 18.246  on 13  degrees of freedom
AIC: 22.246

Number of Fisher Scoring iterations: 4

and no error warning.

I see a contradiction; even in presence of 1 iteration the algorithm does not converge but the output is "finite" (I have not explicitly computed the inverse of the Hessian of the likelihood function, unfortunately). Moreover, with 25 iterations the warning message disappears and the output is still finite.

What do you think about this situation? Is it possible that glm stops automatically after the first iteration? Thank you, Avitus

$\endgroup$
  • $\begingroup$ In the second output, the number of iterations to reach convergence seems to be 4, not 25 or am I missing something? The 25 iterations are the standard maximum interations. If the algorithm didn't converge after 25 iterations, an error message is given. If you set the maximum number of iterations to 1, I don't expect the algorithm to converge. $\endgroup$ – COOLSerdash Jun 3 '13 at 15:37
  • $\begingroup$ @COOLSerdash: this is true, good catch! The algorithm continues to give a finite output even if it diverges, though. I tried also with 10 iterations and the output tells me "number of Fisher int. 4". I do not understand why I get the error warning only for 1 iteration... $\endgroup$ – Avitus Jun 3 '13 at 15:41
  • $\begingroup$ I'm not sure if I can help you but why should the output not be finite? I guess the function just takes some starting values and goes from there. Even if the algorithm doesn't converge, it puts out the values of the last iterations. $\endgroup$ – COOLSerdash Jun 3 '13 at 15:49
  • 1
    $\begingroup$ If the error is printed out, no solution was found and the given values do not represent the maximum. glm uses iterative weighted least squares (IWLS) to find the MLEs. Divergence means that the algorithms jumps around and after the maximum number of iterations, the current values are printed out. In your example, the algorithm converges after 4 iterations. $\endgroup$ – COOLSerdash Jun 3 '13 at 16:23
  • 1
    $\begingroup$ Yes, I think you're right! If the algorithm didn't converge, you can't trust the coefficients, even though they are finite. $\endgroup$ – COOLSerdash Jun 4 '13 at 8:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.