# Why GLM fit is different from direct fit using logistic function [closed]

I found in my data that a direct fit using the logistic function gives a different and better (R^2) fit than GLM fit using binomial distribution with logit link function. I was naively expecting the same results but now can't explain why this is not the case.

As pointed out below in comments it could be due to different minimisation procedures.

Assuming minimisations used by fit and glm are the same does one expect identical fitting results?

Below examples assume the same minimisation procedure.

Here is some matlab code to illustrate the issue

Direct fit

x = [-50.4 -39.6 -29.7 -21.6 -18.0 -14.4 -9.9 -8.1 -6.3 -3.6 -1.8 0.0
1.8 3.6 6.3 8.1 9.9 14.4 18.0 21.6 29.7 39.6 50.4]';
y = [0.0 0.0 0.0 0.0 0.1 0.1 0.2 0.2 0.2 0.3 0.3 0.4 0.5 0.5 0.6 0.6
0.7 0.8 0.9 0.9 1.0 1.0 1.0]';

ft=fittype('exp(b0+b1*x)/(1 + exp(b0+b1*x))', 'indep', 'x');
stp=[-0.5 0.1];
[mo, gf] = fit(x, y, ft, 'StartPoint', stp);

mo =
General model:
mo(x) = exp(b0+b1*x)/(1 + exp(b0+b1*x))
Coefficients (with 95% confidence bounds):
b0 =     -0.4201  (-0.4983, -0.3419)
b1 =      0.1268  (0.1167, 0.1368)


GLM part

data=table(y,x);

modelspec{1}='y ~ x';
mdl1 = fitglm(data, modelspec{1},...
'Distribution', 'binomial', 'Link','logit');

Generalized linear regression model:
logit(y) ~ 1 + x
Distribution = Binomial

Estimated Coefficients:
Estimate       SE        tStat       pValue
________    ________    ________    ________

(Intercept)    -0.42089     0.60619    -0.69432     0.48748
x               0.13401    0.060266      2.2236    0.026175


The difference is not huge but noticeable especially if one expects identical results.

Here is another data example (this one is asymmetric and I know that fit will be by definition symmetric).

y=[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1, 0.5, 1.0,
1.0, 1.0, 1.0, 1.0, 0.7, 0.9, 0.7, 0.8, 0.8, 1.0];


Results

mo =
General model:
mo(x) = exp(b0+b1*x)/(1 + exp(b0+b1*x))
Coefficients (with 95% confidence bounds):
b0 =     0.07416  (-0.793, 0.9413)
b1 =       1.606  (-0.1469, 3.359)

mdl1 =
Generalized linear regression model:
logit(y) ~ 1 + x
Distribution = Binomial

Estimated Coefficients:
Estimate       SE        tStat       pValue
________    ________    ________    ________

(Intercept)    -0.35437     0.61901    -0.57247       0.567
x               0.14666    0.066227      2.2145    0.026791


Here the difference is huge.

Is there any simple explanation as to why the outputs are different? (I check the same data in R using nlsLM and glm and got identical results)

• can you provide details of what fit does? otherwise this is a programming question, and inappropriate here Commented May 28 at 12:54
• i am guessing that fit is doing a least squares fit, but glm is minimising logloss. these weight error differently so will give different answes Commented May 28 at 12:56
• Let me reformulate, if minimisations used by fit and glm are the same do one expect identical results? I used matlab and R default options for both fit and glm. Matlab fit uses Method: 'NonlinearLeastSquares' and Algorithm: 'Trust-Region'. I think matlab fitglm uses least squares as well (not logloss). Commented May 28 at 13:53
• no please check the documentation for fitglm. logloss is the max likelihood metric, it would be bizarre for a glm function to not use max likelihood Commented May 28 at 13:56
• IWLS is not least-squares: it means you iterate least-squares solutions of a non-least squares objetive, basically a second order approximation at each step Commented May 28 at 14:38

## 2 Answers

One of these is a logistic regression model (a generalised linear model) and the other is a nonlinear model fitted by least squares.

I verified this by fitting those two models in R and seeing that they give exactly these results. It's not an optimisation issue; it's a model issue.

For the first example: non-linear least squares

> nls(y~exp(a+b*x)/(1+exp(a+b*x)),data=data.frame(x=x,y=y))
Nonlinear regression model
model: y ~ exp(a + b * x)/(1 + exp(a + b * x))
data: data.frame(x = x, y = y)
a       b
-0.4201  0.1268
residual sum-of-squares: 0.01512


and logistic regression

> glm(y~x, family=binomial())

Call:  glm(formula = y ~ x, family = binomial())

Coefficients:
(Intercept)            x
-0.4209       0.1340


The nonlinear least squares fit finds $$a$$ and $$b$$ to minimise the residual sum of squares $${\text RSS} = \sum_{i=1}^n \left(y_i - \mathrm{expit}(a+bx_i) \right)^2$$ where $$\mathrm{expit}$$ is the function $$x\mapsto \frac{\exp(x)}{1+\exp(x)}$$

The glm finds $$a$$ and $$b$$ to minimise the deviance $$-2\log L= -2\sum_{i=1}^n y_i\log \mathrm{expit} (a+bx_i)+ (1-y_i) \log(1-\mathrm{expit} (a+bx_i))$$

A key difference between these is that nonlinear least squares gives the same weight to all the squared errors but logistic regression weights them according to how close they are to 0 or 1

I found in my data that a direct fit using the logistic function gives a different and better (R^2) fit than GLM fit using binomial distribution with logit link function

There are two seperate issues here. Why is it different? Why is least squares better (in terms of R^2)?