6
$\begingroup$

I am using the package nnet to fit multinomial regression models using multinom().

When fitting the model using an independent variable with much greater mean than variance, the confidence intervals get narrower and the p-value gets smaller. This is unexpected behavior for me.

As a simple univariate example I modified the given example on the multinom() help page. I will ilustrate it using the confint.multinom() function from the same package, since p-values are not natively supported by nnet.

library(MASS)
example(birthwt)
######## Base Line glm logistic Regression #############
glm_model <- glm(formula = low ~ age, family = binomial, data = bwt)
confint(glm_model)

                 2.5 %      97.5 %
(Intercept) -1.0336270 1.847399861
age         -0.1150799 0.008986436
######## Base Line multinomial logistic Regeression ###########
multinom_model <- multinom(formula = low ~ age, data =  bwt)
confint(multinom_model)

# weights:  3 (2 variable)
initial  value 131.004817 
final  value 115.955980 
converged
                 2.5 %    97.5 %
(Intercept) -1.0494438 1.8205227
age         -0.1129621 0.0105759
######## Transformed glm logistic Regression ###########
glm_model_mod <- glm(formula = low ~ I(age+200), family = binomial, data = bwt)
confint(glm_model_mod)

                  2.5 %       97.5 %
(Intercept)  -2.8001518 24.832982980
I(age + 200) -0.1150799  0.008986436
######## Transformed multinomial logistic Regeression ###########
multinom_model_mod <- multinom(low ~ I(age+200), data=bwt)
confint(multinom_model_mod)

# weights:  3 (2 variable)
initial  value 131.004817 
iter  10 value 115.984705
final  value 115.955988 
converged
                   2.5 %      97.5 %
(Intercept)  10.64489882 10.64491130
I(age + 200) -0.05267766 -0.04989515

At first the confidence intervals of multinom are a bit more conservative compared to glm. But when adding 200 to the independent variable, the confidence intervals are getting extremely narrow and even the coefficient for age changes (-0.05119 -> -0.05128). The coefficient for age of the normal logistic regression is, as expected, unaffected by this intercept change.

This example also holds for real multinomial models with a 3 level dependent variable and models with multiple independent variables.

Am I violating model assumptions? Or am I misunderstanding how the multinomial regression works?

$\endgroup$
2
  • $\begingroup$ From the documentation: "multinom calls nnet. The variables on the rhs of the formula should be roughly scaled to [0,1] or the fit will be slow or may not converge at all." It's an important piece of advice to scale inputs to a neural network. $\endgroup$
    – dipetkov
    Jun 28, 2022 at 22:44
  • 1
    $\begingroup$ Thank you for your reply after such a long time. You are right, but my problem is not that convergence fails or is slow. Additionally, the method reports the correct coefficient. It only fails with estimating the correct standard error. And since this method is effectively a logistic regression, there is a correct maximum likelihood solution. Failing to estimate the correct standard error and not giving a warning is definitely a bug. By the way, the package "mgcv" provides a more robust method to fit a multinomial regression. $\endgroup$
    – ndevln
    Jul 2, 2022 at 18:05

1 Answer 1

0
$\begingroup$

In both GLMs and nnet::multinom it is the case that covariates should be scaled to make sure that the fit converges.

So just adding scale() in your model formula's will solve your problem, and will then result in similar SEs on the scaled covariate:

library(MASS)
example(birthwt)
######## Base Line glm logistic Regression #############
glm_model <- glm(formula = low ~ scale(age), family = binomial, data = bwt)
confint(glm_model)
#                  2.5 %      97.5 %
# (Intercept) -1.1232357 -0.49821604
# scale(age)  -0.6097714  0.04761623

######## Base Line multinomial logistic Regeression ###########
library(nnet)
multinom_model <- multinom(formula = low ~ scale(age), data =  bwt)
confint(multinom_model)
#                  2.5 %      97.5 %
# (Intercept) -1.1158831 -0.49234695
# scale(age)  -0.5983204  0.05623451

######## Transformed glm logistic Regression ###########
glm_model_mod <- glm(formula = low ~ scale(I(age+200)), family = binomial, data = bwt)
confint(glm_model_mod)
#                          2.5 %      97.5 %
# (Intercept)         -1.1232357 -0.49821604
# scale(I(age + 200)) -0.6097714  0.04761623

######## Transformed multinomial logistic Regeression ###########
multinom_model_mod <- multinom(low ~ scale(I(age+200)), data=bwt)
confint(multinom_model_mod)
#                          2.5 %      97.5 %
# (Intercept)         -1.1158831 -0.49234695
# scale(I(age + 200)) -0.5983204  0.05623451
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.