I'm fitting a nonlinear dynamic model to some non-normally distributed data using maximum likelihood estimation. My working approach has been to assume my data is gamma distributed and do gamma regression ala something like this: slides on gamma regression.

I'm exploring the possibility of doing a type of Rayleigh distribution regression instead. My model gives an observable response over time in terms of some model parameters: $y(t)=f(t;\theta)$. Suppose I have an observation $x(t)$ at time $t$. Then I have reparameterized the Rayleigh pdf in terms of the expected value $\mu$ as: $$ p(x;\mu) = \frac{\pi}{2}\frac{x}{\mu^2}e^{-\frac{\pi}{4}\frac{x^2}{\mu^2}}, $$ since the scale parameter of the Rayleigh distribution is related to the mean by $\sigma = \mu\sqrt{\frac{2}{\pi}}$.

Now, my understanding of how to do this sort of regression to find MLE's of the model parameters $\theta$ is that I would build a likelihood function for $\theta$ by replacing the expected value of the pdf with the dynamic model dependent variable $y(t;\theta)$. In other words, I would search for values of $\theta$ that maximize the likelihood that the distribution from which observation(s) $x(t)$ are drawn has an expected value (mean) of $y(t) = f(t;\theta)$. Then the likelihood and log-likelihood functions are $$ \mathcal{L}(\theta;x(t)) = \prod_{t} \frac{\pi}{2}\frac{x(t)}{f(t;\theta)^2}e^{-\frac{\pi}{4}\frac{x(t)^2}{f(t;\theta)^2}} $$ and $$ \ln\mathcal{L}(\theta;x(t)) = \sum_t \frac{1}{2}\ln\frac{\pi}{2} + \ln x(t) - 2\ln f(t;\theta) - \frac{\pi}{4}\frac{x(t)^2}{f(t;\theta)^2}. $$

I then solve the MLE probelm for $\theta$ my minimizing the negative log likelihood above.

My problem is that the deviances that come out of MLE's w.r.t. this log likelihood function are sometimes negative! My understanding is that the saturated log likelihood should be $$ \ln\mathcal{L}_{sat} = \sum_t \frac{1}{2}\ln\frac{\pi}{2} + \ln x(t) - 2\ln x(t) - \frac{\pi}{4}\frac{x(t)^2}{x(t)^2} = \sum_t \frac{1}{2}\ln\frac{\pi}{2} - \ln x(t) -\frac{\pi}{4}, $$ and then the deviance for an MLE of $\theta$ at $x(t)$ will be: $$ \begin{split} \mathcal{D}(\hat{\theta};x(t)) &= -2(\ln\mathcal{L}(\hat{\theta};x(t)) -\ln\mathcal{L}_{sat}) \\ &=-2\left( - 2\ln f(t;\hat{\theta}) + 2\ln(x(t)) - \frac{\pi}{4}\frac{x(t)^2}{f(t;\hat{\theta})^2} +\frac{\pi}{4} \right) \\ &= -2\left( 2\ln\frac{x(t)}{f(t;\hat{\theta})} + \frac{\pi}{4}\left(1-\frac{x(t)^2}{f(t;\hat{\theta})^2}\right) \right). \end{split} $$

This function can be negative for certain values of $x$ and $f$, i.e. for $x=5$ and $f=4.5$. But it seems to me that the deviance of the fitted model relative to the saturated model should always be positive?

Side Note: It turns out that if you replace $\frac{\pi}{4}$ with $1$ in the above equation for deviances, then the deviances are always positive. This leads me to question whether or not the reparameterization in terms of mean that I'm using is legitimate for the MLE problem?

  • $\begingroup$ There seems to be some less than rigorous back-and-forth between performing the sums and not performing them, causing factors of $n$ to be dropped in various places. $\endgroup$ – whuber Mar 23 '17 at 19:20
  • $\begingroup$ Ah, yes... I was a bit hand-wavey with the summands. The sums in the log-likelihood are really only used for the MLE problem... while the deviance should generally be defined as summed over all observations, my concern is more along the lines that I think the deviance of the MLE solution from individual observations should be chi^2 distributed. $\endgroup$ – K Ball Mar 23 '17 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.