1
$\begingroup$

I have a group of items. I model each with the Poisson regression (counts), but one of the regression coefficients is modelled on a group level using the normal distribution.

Assuming there are $m$ items, and for each item, there are $n$ data points. The observation $i$ for the item $j$ is denoted as $y_{ji}$, the features are $x_{ji}$, the $j$'s item distribution parameters independent from other items are $\theta_{j}$, and the parameter drawn from a common group is $\phi_j$. Then the probability density of the item is:

$$\prod_{i=1}^nPois(y_{ji}|x_{ji}, \theta_j, \phi_j)$$

Next, each coefficient $\phi_j$ comes from the common normal distribution:

$$p(\phi_j| a, b) = \frac{b}{2\pi}e^{-\frac{b(\phi_j - a)}{2}}$$

where $a$ is the mean, and $b$ is the precision.

Thus, the full hierarchical model can be written as:

$$p(Y | X) = \prod_{j = 1}^m\bigg\{\frac{b}{2\pi}e^{-\frac{b(\phi_j - a)}{2}}\prod_{i=1}^nPois(y_{ji}|x_{ji}, \theta_j, \phi_j)\bigg\}$$

I need to find the parameters $\theta_j, \phi_j, a, b$. I use MLE for this task.

In order to optimize the whole thing, I use the Newton method and coordinate ascent. It works by optimizing each item in turn, and then estimate the normal group-level parameters $a$ and $b$ analytically.

However, the log-likelihood optimization somehow sets the parameters $\phi_j$ for ALL items to THE SAME VALUE. As a result, the precision tends to infinity, and MLE fails! What process can cause this model to set all coefficients $\phi_j$ to the same value? Other parameters, $\theta_j$, and the group mean $a$ seem to be optimized correctly (look reasonable to me).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.