# Derivation of a marginal log likelihood

I have a question concerning a derivation of a marginal log-Likelihood function that I found in a paper and that I do not understand. It's pretty basic so sorry for this in advance.

Assume that the marginal Likelihood of object $i$ is given by

$f(e_i) = \int_{\theta} f(e_i|\theta)g(\theta)\, d\theta$

and the Likelihood for $N$ objects is $L = \Pi_{i = 1}^{N} f(e_i)$. Set $f(e_i) = f_i, c_i = f(e_i|\theta)$, and $g(\theta) = g$.

We are interested in estimating parameters that are in $c_i$. It may be important that $\theta$ is a two-dimensional vector. We therefore consider the log-Likelihood given by

$\ln L = \sum_{i=1}^{N} \ln(f_i) = \sum_{i=1}^{N} \ln \left[ \int_{\theta} (c_i \cdot g )\,d\theta \right]$

In the paper they say that the first derivative of $\ln L$ concerning a parameter vector $\eta$ is given by

$\frac{\partial \ln L}{\partial \eta} = \sum_{i=1}^{N} \frac{1}{f_i} \int_{\theta} \left[ \frac{\partial \ln(f_i)}{\partial \eta} \cdot f_i \cdot g \right] \, d\theta$

My question is how one arrives at this derivation. The first part of the derivation, i.e., $\frac{1}{f_i}$ seems to be an application of the chain rule. But in this case one would also need the derivation of

$\frac{\partial }{\partial \eta} \int_{\theta} f(e_i|\theta)g(\theta) \, d\theta$ which does not seem to involve another logarithm so I would not end up with $\frac{\partial \ln(f_i)}{\partial \eta}$ within the integral. Are the authors using another rule? I would be happy, if you could give me a hint.

• $\eta$? Where is $\eta$ defined? I only see it appear when you state 'concerning a parameter vector $\eta$', and not earlier? Commented Dec 9, 2017 at 22:59
• It's somewhat confusing when you change notation part way through the question, probably best not to do so unless you have a very good reason. Commented Dec 10, 2017 at 1:55
• Sorry for changing the notation. I did not catch this when I read the question prior to sending it to the forum. Commented Dec 11, 2017 at 9:01

Let's try to clear up some of the notation and simplify the problem. We'll return to your initial notation, which is a lot clearer, and extend it by showing where $\eta$ fits in:

$$\ln L = \Sigma_i \ln f(e_i|\eta) = \Sigma_i \ln \int_{\theta}f(e_i|\eta, \theta)g(\theta)d\theta$$

I'm making the simplifying assumption that $\eta$ is a parameter of the conditional distribution.

Now, a bit of notational simplification. Note that the sum over $i = 1\dots N$ doesn't matter; if you have the right expression for $i=1$, you'll have it right for all the other values of $i$ too, because they are all the same except for the index. So we can drop the summation and index altogether.

As written, the equation which is causing you trouble doesn't seem very useful, and I suspect you actually meant something else. What you have written translates into:

$$\frac{\partial \ln L}{\partial \eta} = \frac{1}{f(e|\eta)} \int_{\theta}\frac{\partial \ln f(e|\eta)}{\partial \eta}f(e|\eta)g(\theta)d\theta$$

Observe that the first two terms under the integral are not functions of $\theta$. They can therefore be moved outside the integral, giving:

$$\frac{\partial \ln L}{\partial \eta} = \frac{1}{f(e|\eta)} \frac{\partial \ln f(e|\eta)}{\partial \eta}f(e|\eta)\int_{\theta}g(\theta)d\theta$$

Making the obvious cancellation and observing that $\int_{\theta}g(\theta)d\theta = 1$ enables us to simplify the above:

$$\frac{\partial \ln L}{\partial \eta} = \frac{\partial \ln f(e|\eta)}{\partial \eta}$$

which would hardly seem worth the effort of expanding to the original form.

On to the problem that I think you are trying to understand. We start with the derivative, which can be written as:

$$\frac{\partial \ln L}{\partial \eta} = \frac{\partial \ln \int_{\theta}f(e|\eta, \theta)g(\theta)d\theta}{\partial \eta}$$

Using the chain rule gives:

$$\frac{1}{\int_{\theta}f(e|\eta, \theta)g(\theta)d\theta} \frac{\partial \int_{\theta}f(e|\eta, \theta)g(\theta)d\theta}{\partial \eta}$$

Substituting $f(e|\theta)$ for the integral in the denominator of the first term and reversing the order of differentiation and integration results in:

$$\frac{1}{f(e|\eta)}\int_{\theta}\frac{\partial f(e|\eta, \theta)}{\partial \eta}g(\theta)d\theta$$

Now for a minor sleight of hand. We observe that:

$$\frac{\partial f(e|\eta, \theta)}{\partial \eta} = \frac{\partial \ln f(e|\eta, \theta)}{\partial \eta} f(e|\eta, \theta)$$

To see this, just divide both sides by $f(e|\eta, \theta)$. Substitution gives us:

$$\frac{\partial \ln L}{\partial \eta} = \frac{1}{f(e|\eta)}\int_{\theta}\frac{\partial \ln f(e|\eta, \theta)}{\partial \eta} f(e|\eta, \theta)g(\theta)d\theta$$

which is what I think you may have meant to write.

• Thanks jbowman. I have not stated the concrete densities in the question above, but the whole rewriting seems to be a "trick" as the derivation of the log of the conditional density is simpler than the derivation of the conditional density itself. I have not get that. Thanks again! Commented Dec 11, 2017 at 9:07
• Well, it is a trick, in the math sense of the word. Often the derivative of the log of the likelihood function is much easier to work with than the derivative of the likelihood function itself, which is what makes it a worthwhile trick! Commented Dec 11, 2017 at 15:30