Can we instead of doing logistic or multinomial regression do two or multiple Poisson regressions and then combine Poisson predictions to get probabilistic predictions? If yes, how should we transform data for training?


Assume that you're fitting a multinomial regression model with J categories where the contrast between j and the last category J is modeled as $$ \log \frac{\mu_{ij}}{\mu_{iJ}} = \alpha_j + X_i \beta_j $$ where $X_i$ is a vector of covariates associated with the $i$th case. This is rather a hard optimisation problem because the parameters are coupled by the multinomial's conditioning on $N_i$, the $i$-th case's marginal total.

You can instead fit J separate Poisson regressions with linear predictor of the form $$ \log\, \mu_{ij} = \eta + \theta_i + \alpha_j^* + X_i\beta_j^* $$ where each case gets one nuisance parameter $\theta$.

Subtracting the expression for $\log\, \mu_{ij}$ from $\log\, \mu_{iJ}$ to construct $\log (\mu_{ij}/\mu_{iJ})$ shows that the parameters are related as $\alpha_j = \alpha^*_j − \alpha^*_J$ and $\beta_j = \beta^*_j - \beta^*_J$ (the nuisance parameters and $\eta$ cancel). Thus you get what you came for: fitting multiple Poisson regression models gives you the same result as fitting a single more complicated multinomial regression model.

This exposition is essentially the one in Gérman Rodríguez's always excellent notes, in the section on the 'equivalent log-linear model'.

In general this manoeuvre is called the multinomial-Poisson transformation and was written about by Baker in 1994.

Notice also that you don't need to adjust $X$ at all to use it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.