I have a group of items. I model each with the Poisson regression (counts), but one of the regression coefficients is modelled on a group level using the normal distribution.

Assuming there are $m$ items, and for each item, there are $n$ data points. The observation $i$ for the item $j$ is denoted as $y_{ji}$, the features are $x_{ji}$, the $j$'s item distribution parameters independent from other items are $\theta_{j}$, and the parameter drawn from a common group is $\phi_j$. Then the probability density of the item is:

$$\prod_{i=1}^nPois(y_{ji}|x_{ji}, \theta_j, \phi_j)$$

Next, each coefficient $\phi_j$ comes from the common normal distribution:

$$p(\phi_j| a, b) = \frac{b}{2\pi}e^{-\frac{b(\phi_j - a)}{2}}$$

where $a$ is the mean, and $b$ is the precision.

Thus, the full hierarchical model can be written as:

$$p(Y | X) = \prod_{j = 1}^m\bigg\{\frac{b}{2\pi}e^{-\frac{b(\phi_j - a)}{2}}\prod_{i=1}^nPois(y_{ji}|x_{ji}, \theta_j, \phi_j)\bigg\}$$

I need to find the parameters $\theta_j, \phi_j, a, b$. I use MLE for this task.

In order to optimize the whole thing, I use the Newton method and coordinate ascent. It works by optimizing each item in turn, and then estimate the normal group-level parameters $a$ and $b$ analytically.

However, the log-likelihood optimization somehow sets the parameters $\phi_j$ for ALL items to THE SAME VALUE. As a result, the precision tends to infinity, and MLE fails! What process can cause this model to set all coefficients $\phi_j$ to the same value? Other parameters, $\theta_j$, and the group mean $a$ seem to be optimized correctly (look reasonable to me).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.