Marginal Distribution of Exponential Mixture Model

Question

I am currently trying to marginalize over the scale parameter in a mixture distribution of exponential pdfs, but I do not trust my result. Let me show you my steps:

Probability Density Function

The pdf for my exponential mixture distribution is given by:

$$ prob(t|\{a_j\},\{\tau_j\})=\sum_j a_j \cdot 1/\tau_j \cdot \exp(-t/\tau_j)$$

It describes the probability of observing an event (i.e. a radioactive decay) at time $t\in [0,\infty)$, given the scale parameters $\tau_j$ and the relative contributions $a_j$ of different decay components. By normalization we require $\sum_j a_j =1$. Futhermore let $j=1,...,N$, so that we have $N$ distinct components

Marginalization Integral

To get rid of the scale parameters I carried out the marginalization like this: $$\begin{eqnarray} prob(t|\{a_j\})&=&\int d\tau_1...\int d\tau_N \quad prob(t,\{\tau_j\}|{a_j}) \\ &=&\int d\tau_1...\int d\tau_N \quad prob(t|\{a_j\},\{\tau_j\})\cdot \prod_j prob(\tau_j) \end{eqnarray}$$ where I have made the assumption that the priors of any two scale parameters are independent of each other and independent of the amplitudes.

Assigning the Prior Distribution

For the prior distribution of the scale parameter I assume knowlegde that $\tau_j \in [\tau_j^{min},\tau_j^{max}]$ but otherwise complete ignorance, which is (I think) expressed by using the Jeffreys Prior, i.e.

$prob(\tau_j)=ln(\tau^{max}_j/\tau^{min}_j)\cdot 1/\tau_j$ for $\tau_j \in [\tau_j^{min},\tau_j^{max}]$ and $0$ otherwise.

The $ln(...)$ term I have included for normalization.

Revisiting the Marginalization Integral

Since the prior distributions are normalized to $1$, they only contribute when integrating over terms that are dependent on the specific $\tau_j$. Thus the important integrals to carry out for each term in the sum are: $$I_j(t) := \int_{ \tau_j^{min}} ^{ \tau_j^{max}} ln(\tau^{max}_j/\tau^{min}_j)\cdot 1/\tau_j^2\cdot \exp(-t/\tau_j)$$ this becomes (use Wolfram Alpha to check my result): $$I_j(t) = ln(\tau^{max}_j/\tau^{min}_j)\cdot \left(\frac{\exp(-t/\tau_j^{max})}{t} - \frac{\exp(-t/\tau_j^{min})}{t}\right)$$ and finally the marginalized distribution should be $$prob(t|\{a_j\})=\sum_j a_j \cdot I_j(t)$$

My Question

My question is simply: is this correct? I do not trust the result because I cannot see the term $I_j$ converge to the original exponential in the limit that I know the scale parameter, i.e. $\tau_j^{min}\rightarrow \tau_j^{max}$.

Did I make an error in my assumptions or calculations?

Does it converge to the desired exponential after all? Or am I wrong in assuming that it should do that at all?

All help is greatly appreciated.

Answer 1

This is indeed correct, modulo a minor point, when observing a single realisation from the mixture. But not when observing an $n$-sample from the same mixture since the integrals in the $\tau_j$'s are over the product of the mixture sums, which involves $N^n$ terms, hence quickly gets intractable. The reasoning is that all the points in the sample "share" the same $\tau_j$'s. Hence, if $$X_1,X_2\stackrel{\text{i.i.d.}}{\sim} f(x|\tau)$$ the marginal distribution of $(X_1,X_2)$ is $$\int_T f(x_1|\tau)f(x_2|\tau)\text{d}\tau\quad\text{not}\quad \int_T f(x_1|\tau)\text{d}\tau\times\int_T f(x_2|\tau)\text{d}\tau$$

The minor point is that the normalisation for the prior should be $$\ln(\tau^{max}_j/\tau^{min}_j)^{-1}$$ which makes the limit of $$\ln(\tau^{max}_j/\tau^{min}_j)^{-1}\cdot \left(\frac{\exp(-t/\tau_j^{max})}{t} - \frac{\exp(-t/\tau_j^{min})}{t}\right)$$ when $\tau^{min}_j\to\tau^0_j$ and $\tau^{max}_j\to\tau^0_j$ equal to the limit of $$\underbrace{\dfrac{\tau^{max}_j-\tau^{min}_j}{\ln\tau^{max}_j-\ln\tau^{min}_j}}_\text{inverse derivative of $\ln(\tau)$}\cdot \underbrace{\dfrac{\exp(-t/\tau_j^{max})- \exp(-t/\tau_j^{min})}{\tau^{max}_j-\tau^{min}_j}}_\text{derivative of $\exp(-t/\tau)$}\cdot\frac{1}{t}$$when $\tau^{min}_j\to\tau^0_j$ and $\tau^{max}_j\to\tau^0_j$ hence equal to the ratio of the derivatives of the two functions of $\tau$ at $\tau^0_j$: $$\dfrac{1}{\frac{1}{\tau^0_j}}\cdot \dfrac{t\exp(-t/\tau_j^0)}{{\tau^{0}_j}^2}\cdot\frac{1}{t}=\exp(-t/\tau_j^0)\big/{\tau_j^0}$$ recovering $\exp\{-t/\tau_j^0\}/\tau_j^0$.

Note also that $$\dfrac{\exp(-t/\tau_j^{max}) - \exp(-t/\tau_j^{min})}{t}$$ does not diverge at zero since, by L'Hospital's rule \begin{align*}\lim_{t\to 0} \dfrac{\exp(-t/\tau_j^{max}) - \exp(-t/\tau_j^{min})}{t} &=\lim_{t\to 0} \dfrac{\frac{\text{d}}{\text{d}t}\left[\exp(-t/\tau_j^{max}) - \exp(-t/\tau_j^{min})\right]}{\frac{\text{d}}{\text{d}t}t}\\ &=\lim_{t\to 0} \dfrac{-\exp(-t/\tau_j^{max})/\tau_j^{max} + \exp(-t/\tau_j^{min})/\tau_j^{min}}{1}\\ &=\frac{1}{\tau_j^{min}}-\frac{1}{\tau_j^{max}} \end{align*}