# Marginal Distribution of Exponential Mixture Model

I am currently trying to marginalize over the scale parameter in a mixture distribution of exponential pdfs, but I do not trust my result. Let me show you my steps:

Probability Density Function

The pdf for my exponential mixture distribution is given by:

$$prob(t|\{a_j\},\{\tau_j\})=\sum_j a_j \cdot 1/\tau_j \cdot \exp(-t/\tau_j)$$

It describes the probability of observing an event (i.e. a radioactive decay) at time $$t\in [0,\infty)$$, given the scale parameters $$\tau_j$$ and the relative contributions $$a_j$$ of different decay components. By normalization we require $$\sum_j a_j =1$$. Futhermore let $$j=1,...,N$$, so that we have $$N$$ distinct components

Marginalization Integral

To get rid of the scale parameters I carried out the marginalization like this: $$\begin{eqnarray} prob(t|\{a_j\})&=&\int d\tau_1...\int d\tau_N \quad prob(t,\{\tau_j\}|{a_j}) \\ &=&\int d\tau_1...\int d\tau_N \quad prob(t|\{a_j\},\{\tau_j\})\cdot \prod_j prob(\tau_j) \end{eqnarray}$$ where I have made the assumption that the priors of any two scale parameters are independent of each other and independent of the amplitudes.

Assigning the Prior Distribution

For the prior distribution of the scale parameter I assume knowlegde that $$\tau_j \in [\tau_j^{min},\tau_j^{max}]$$ but otherwise complete ignorance, which is (I think) expressed by using the Jeffreys Prior, i.e.

$$prob(\tau_j)=ln(\tau^{max}_j/\tau^{min}_j)\cdot 1/\tau_j$$ for $$\tau_j \in [\tau_j^{min},\tau_j^{max}]$$ and $$0$$ otherwise.

The $$ln(...)$$ term I have included for normalization.

Revisiting the Marginalization Integral

Since the prior distributions are normalized to $$1$$, they only contribute when integrating over terms that are dependent on the specific $$\tau_j$$. Thus the important integrals to carry out for each term in the sum are: $$I_j(t) := \int_{ \tau_j^{min}} ^{ \tau_j^{max}} ln(\tau^{max}_j/\tau^{min}_j)\cdot 1/\tau_j^2\cdot \exp(-t/\tau_j)$$ this becomes (use Wolfram Alpha to check my result): $$I_j(t) = ln(\tau^{max}_j/\tau^{min}_j)\cdot \left(\frac{\exp(-t/\tau_j^{max})}{t} - \frac{\exp(-t/\tau_j^{min})}{t}\right)$$ and finally the marginalized distribution should be $$prob(t|\{a_j\})=\sum_j a_j \cdot I_j(t)$$

My Question

My question is simply: is this correct? I do not trust the result because I cannot see the term $$I_j$$ converge to the original exponential in the limit that I know the scale parameter, i.e. $$\tau_j^{min}\rightarrow \tau_j^{max}$$.

Did I make an error in my assumptions or calculations?

Does it converge to the desired exponential after all? Or am I wrong in assuming that it should do that at all?

All help is greatly appreciated.

• Your integral in "Revisiting the Marginalization Integral" should run from $\tau_j^{min}$ to $\tau_j^{max}$, and there's no need for the $\ln$ term, as it's a constant so will disappear when the final constant of integration is calculated. – jbowman Oct 16 '18 at 17:20
• Thank you, I mistyped. I corrected the borders of integration. The result however was calculated using the correct borders. – geo Oct 16 '18 at 17:31
• Maybe you are on to something with the borders of integration. Should I try and normalize the whole integral again? I was under the assumption that it would be fine since I merely used the product rule – geo Oct 16 '18 at 17:35
• @jbowman You really touched on a weak spot there. Come to think of it: Normalization of my marginal distribution in $t\in[0,\infty]$ really seems to be the Achilles heel of my approach, because the distribution diverges at zero t. Why would that be the case? What can I do about it? – geo Oct 16 '18 at 19:05
• I've come to the same conclusion, and it doesn't change if you alter your prior slightly, e.g., to $\tau^{-1-\alpha}$. I might try making the distributions of $t$ Gamma with shape parameter $a$, solve, and see what happens when $a \rightarrow 1$ (from above); the limit of the distributions may not be the same as the distribution at the limit. – jbowman Oct 16 '18 at 19:27

This is indeed correct, modulo a minor point, when observing a single realisation from the mixture. But not when observing an $$n$$-sample from the same mixture since the integrals in the $$\tau_j$$'s are over the product of the mixture sums, which involves $$N^n$$ terms, hence quickly gets intractable. The reasoning is that all the points in the sample "share" the same $$\tau_j$$'s. Hence, if $$X_1,X_2\stackrel{\text{i.i.d.}}{\sim} f(x|\tau)$$ the marginal distribution of $$(X_1,X_2)$$ is $$\int_T f(x_1|\tau)f(x_2|\tau)\text{d}\tau\quad\text{not}\quad \int_T f(x_1|\tau)\text{d}\tau\times\int_T f(x_2|\tau)\text{d}\tau$$
The minor point is that the normalisation for the prior should be $$\ln(\tau^{max}_j/\tau^{min}_j)^{-1}$$ which makes the limit of $$\ln(\tau^{max}_j/\tau^{min}_j)^{-1}\cdot \left(\frac{\exp(-t/\tau_j^{max})}{t} - \frac{\exp(-t/\tau_j^{min})}{t}\right)$$ when $$\tau^{min}_j\to\tau^0_j$$ and $$\tau^{max}_j\to\tau^0_j$$ equal to the limit of $$\underbrace{\dfrac{\tau^{max}_j-\tau^{min}_j}{\ln\tau^{max}_j-\ln\tau^{min}_j}}_\text{inverse derivative of \ln(\tau)}\cdot \underbrace{\dfrac{\exp(-t/\tau_j^{max})- \exp(-t/\tau_j^{min})}{\tau^{max}_j-\tau^{min}_j}}_\text{derivative of \exp(-t/\tau)}\cdot\frac{1}{t}$$when $$\tau^{min}_j\to\tau^0_j$$ and $$\tau^{max}_j\to\tau^0_j$$ hence equal to the ratio of the derivatives of the two functions of $$\tau$$ at $$\tau^0_j$$: $$\dfrac{1}{\frac{1}{\tau^0_j}}\cdot \dfrac{t\exp(-t/\tau_j^0)}{{\tau^{0}_j}^2}\cdot\frac{1}{t}=\exp(-t/\tau_j^0)\big/{\tau_j^0}$$ recovering $$\exp\{-t/\tau_j^0\}/\tau_j^0$$.
Note also that $$\dfrac{\exp(-t/\tau_j^{max}) - \exp(-t/\tau_j^{min})}{t}$$ does not diverge at zero since, by L'Hospital's rule \begin{align*}\lim_{t\to 0} \dfrac{\exp(-t/\tau_j^{max}) - \exp(-t/\tau_j^{min})}{t} &=\lim_{t\to 0} \dfrac{\frac{\text{d}}{\text{d}t}\left[\exp(-t/\tau_j^{max}) - \exp(-t/\tau_j^{min})\right]}{\frac{\text{d}}{\text{d}t}t}\\ &=\lim_{t\to 0} \dfrac{-\exp(-t/\tau_j^{max})/\tau_j^{max} + \exp(-t/\tau_j^{min})/\tau_j^{min}}{1}\\ &=\frac{1}{\tau_j^{min}}-\frac{1}{\tau_j^{max}} \end{align*}
• Thank you a lot. Although I had the $ln(...)^{-1}$ correct in my notes, I could not see this very elegant solution. I have further questions: * Could you elaborate on what you mean when you say that it is not OK to use this distribution as the basis of observing $n$ samples? Is it wrong to assume any two samples independent and assign the same probability distribution to them? * Why does the distribution diverge at $t=0$? What can I do about normalization? I have tried using the gamma function instead of the exponential (as suggested above by @jbowman). Unfortunately that didn't do the trick – geo Oct 18 '18 at 17:41
• Thank you. I missed the part on the limit in $t \rightarrow 0$. The answer in (i) I still do not understand. Could you point me to literature where I could read up on the matter? I was under the impression that I now have a likelihood that describes the probability of observing an event at time t that incorporates my prior knowledge on the decay rates $\tau_j$. So my likelihood $prob(\{tau_m\}|\{a_j\})$ of observing a set of time points $\{t_m\}$ could be described by a product like $\prod_m prob(t_m|\{a_j\})$. I gladly appreciate any hints. Thank you again. – geo Oct 18 '18 at 17:58