Skip to main content
added 91 characters in body
Source Link
AdamO
  • 64.8k
  • 6
  • 135
  • 273

So to answer OP's question, the reason why these are irreconcilable results (despite the converge fail) is that the actual trace of the Fisher Scoring differs for weighted and unweighted analyses because in the weighted case, the Fisher information is based on the 3 observation weighted sample, in the unweighted case, the Fisher Information is based on the 60 observation unweighted information. TheyThe 3 observation weighted and 60 observation unweighted likelihoods only agree when Fisher scoring actually obtains a beta estimate giving a 0 scoresum-score solution.

So to answer OP's question, the reason why these are irreconcilable results (despite the converge fail) is that the actual trace of the Fisher Scoring differs for weighted and unweighted analyses because in the weighted case, the Fisher information is based on the 3 observation weighted sample, in the unweighted case, the Fisher Information is based on the 60 observation unweighted information. They only agree when Fisher scoring actually obtains a 0 score solution.

So to answer OP's question, the reason why these are irreconcilable results (despite the converge fail) is that the actual trace of the Fisher Scoring differs for weighted and unweighted analyses because in the weighted case, the Fisher information is based on the 3 observation weighted sample, in the unweighted case, the Fisher Information is based on the 60 observation unweighted information. The 3 observation weighted and 60 observation unweighted likelihoods only agree when Fisher scoring actually obtains a beta estimate giving a 0 sum-score solution.

Source Link
AdamO
  • 64.8k
  • 6
  • 135
  • 273

Despite the converge fail that was illustrated in this example, it should be noted there are indeed some key differences in these applications. A weighted GLM has number of observations equal to the number of response levels, even when the weights are frequency weights. On the other hand, if you replicate the factor levels according to the frequency weights, the number of observations is equal to the sum of the weights (appropriately). Ultimately, they will converge to the same thing, but interesting behavior is observed when you inspect the properties of the one-step estimators:

set.seed(123)
x <- 0:2
y <- c(1,0,2)/2
w <- 1:3*10

## weighted and unweighted one step glms
summary(glm(y ~ x, family=binomial, weights=w, control=list(maxit = 1)))
summary(glm(y ~ x, family=binomial, data.frame('y'=rep.int(y, w), 'x'=rep.int(x,w)), control=list(maxit = 1)))

Give the following (different) results:

Call:
glm(formula = y ~ x, family = binomial, weights = w, control = list(maxit = 1))

Deviance Residuals: 
      1        2        3  
 0.8269  -7.0855   2.3210  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)  
(Intercept)  -0.5260     0.6210  -0.847   0.3970  
x             1.4456     0.7484   1.932   0.0534 .
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 67.640  on 2  degrees of freedom
Residual deviance: 56.275  on 1  degrees of freedom
AIC: 63.079

Number of Fisher Scoring iterations: 1

Warning message:
glm.fit: algorithm did not converge 
> 

Call:
glm(formula = y ~ x, family = binomial, data = data.frame(y = rep.int(y, 
    w), x = rep.int(x, w)), control = list(maxit = 1))

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.1496  -1.1496   0.5946   0.5946   0.8376  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  -1.7747     0.5502  -3.226  0.00126 ** 
x             1.7089     0.3700   4.618 3.87e-06 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 67.640  on 59  degrees of freedom
Residual deviance: 44.055  on 58  degrees of freedom
AIC: 44.171

Number of Fisher Scoring iterations: 1

Warning messages:
1: In eval(expr, envir, enclos) :
  non-integer #successes in a binomial glm!
2: glm.fit: algorithm did not converge 
> 

So to answer OP's question, the reason why these are irreconcilable results (despite the converge fail) is that the actual trace of the Fisher Scoring differs for weighted and unweighted analyses because in the weighted case, the Fisher information is based on the 3 observation weighted sample, in the unweighted case, the Fisher Information is based on the 60 observation unweighted information. They only agree when Fisher scoring actually obtains a 0 score solution.