2
$\begingroup$

Following the wikipedia article on multinomial logistic regression i do not fully understand why how to get from the "independent binary regressions" to the "log-linear model".

At school we discussed what wikipedia calls the approach of "independent binary regressions". With the probabilities

\begin{aligned}\Pr(Y_{i}=1)&={\frac {e^{{\boldsymbol {\beta }}_{1}\cdot \mathbf {X} _{i}}}{1+\sum _{k=1}^{K-1}e^{{\boldsymbol {\beta }}_{k}\cdot \mathbf {X} _{i}}}}\\\Pr(Y_{i}=2)&={\frac {e^{{\boldsymbol {\beta }}_{2}\cdot \mathbf {X} _{i}}}{1+\sum _{k=1}^{K-1}e^{{\boldsymbol {\beta }}_{k}\cdot \mathbf {X} _{i}}}}\\\cdots &\cdots \\\Pr(Y_{i}=K-1)&={\frac {e^{{\boldsymbol {\beta }}_{K-1}\cdot \mathbf {X} _{i}}}{1+\sum _{k=1}^{K-1}e^{{\boldsymbol {\beta }}_{k}\cdot \mathbf {X} _{i}}}}\\\end{aligned}

But i am not sure how this compares to the "log-linear-model" with

\begin{aligned}\Pr(Y_{i}=1)&={\frac {1}{Z}}e^{{\boldsymbol {\beta }}_{1}\cdot \mathbf {X} _{i}}\,\\\Pr(Y_{i}=2)&={\frac {1}{Z}}e^{{\boldsymbol {\beta }}_{2}\cdot \mathbf {X} _{i}}\,\\\cdots &\cdots \\\Pr(Y_{i}=K)&={\frac {1}{Z}}e^{{\boldsymbol {\beta }}_{K}\cdot \mathbf {X} _{i}}\,\\\end{aligned}

Starting from the model

\begin{aligned} \ln \Pr(Y_{i}=2)&={\boldsymbol {\beta }}_{2}\cdot \mathbf {X} _{i}-\ln Z \end{aligned}

I can follow it mathematically but how do i motivate this model? I found http://data.princeton.edu/wws509/notes/c6.pdf, https://www.quora.com/What-is-the-relationship-between-Log-Linear-model-MaxEnt-model-and-Logistic-Regression and Multinomial logistic regression vs one-vs-rest binary logistic regression but still i am not sure how they compare...

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.