My question concerns the calculation of a marginal likelihood given some priors for my underlying exponential mixture distribution.


My data is the (orderd) set of integers $\{N_\ell\}$. My underlying model leads to an expected $\mu_\ell$ value for each bin

$$\mu_\ell(a_1,a_2,\tau_1,\tau_2) = \frac{a_1}{\tau_1}\exp(-t_\ell/\tau_1) + \frac{a_2}{\tau_2}\exp(-t_\ell/\tau_2) .$$ $a_{1,2}$ are the amplitudes of the decay components and $t_{1,2}$ their respective decay times. I want to find these given the data with MLE.

My resulting likelihood of observing the set $\{N_\ell\}$ by Gaussian approximation is:

$$\begin{eqnarray} prob(\{N_\ell\}|a_1,a_2,\tau_1,\tau_2) &\propto& \prod_{\ell} \exp\left(-\frac{(\mu_\ell(a_1,a_2,\tau_1,\tau_2)-N_\ell)^2}{2N_\ell}\right) \\ &=&\exp\left(-\sum_\ell \frac{(\mu_\ell(a_1,a_2,\tau_1,\tau_2)-N_\ell)^2}{2N_\ell}\right) \end{eqnarray}$$ Now maximizing the log likelihood would lead me to Least Squares Regression.

What I hoped to achieve

I have a fairly good idea of the ranges of $\tau_1$ and $\tau_2$. Those parameters are what make the Least Squares Regression nonlinear and somewhat unstable. Thus hoping to get rid of them I want to calculate the marginal likelihood, i.e. $$prob(\{N_\ell\}|a_1,a_2) \propto \int d\tau_1 \int d\tau_2 \,prob(\{N_\ell\}|a_1,a_2,\tau_1,\tau_2) prob(\tau_1) prob(\tau_2) .$$

Assume I can give an analytical expression for the priors. They could be uniform (in a range), Gaussian or Jeffrey's (in a range).

My Questions

  1. Hardest Question: What is an efficient way of calculating the coefficients $a_1$,$a_2$ from the marginal likelihood? By efficient I mean: computationally faster and more stable than performing the whole nonlinear regression of $a_1$,$a_2$,$\tau_1$,$\tau_2$ on the original likelihood?
  2. Can the marginal log likelihood be approximated from the integral?
  3. Is my idea useful at all? Maybe I am naive in assuming that marginalization over the decay times can just magically lead me to a simpler solution.

What I have tried

  • Analytical integration: Even after rearranging I cannot se a way of dealing with the integrals of $ e^{e^{1/\tau}}$.
  • Evaluating the marginal likelihood (not log likelihood) with numerical integration (using OCTAVE): Quickly gave up because everything evaluated to zero, of course.
  • Using Laplace's Approximation: I considered it, but did not give it a try, since it effectively requires me to solve the minimization problem for $\tau_1$,$\tau_2$ if I understood correctly. If I do this, I might just as well go with the whole nonlinear least squares from the start. Maybe I am wrong?

As always any help is greatly appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.