I've got a simple question about deriving log-likelihoods. I am stumped by the following-->

If the log-likelihood is: 𝑙(𝜆1,𝜆2) = 𝑦1 log(𝜆1𝐹1)−𝜆1𝐹1 −log((𝑦1)!)+𝑦2 log(𝜆2𝐹2) −𝜆2𝐹2 −log⁡ ((𝑦2)!)

Then if we omit terms not depending on 𝜆𝑖 the log-likelihood becomes: 𝑙(𝜆1,𝜆2) = 𝑦1 log(𝜆1)−𝜆1𝐹1 +𝑦2 log(𝜆2)−𝜆2𝐹2

I am confused why we're able to drop the F1 and F2 from the log expressions? Aren't they dependent on 𝜆𝑖? Am I just forgetting a simple log law...?

  • $\begingroup$ You do: log(ab)=log(a)+log(b)... $\endgroup$ – Fabian Werner Aug 26 '18 at 12:52
  • $\begingroup$ Thank you, though I'm still lost. Why do F1 and F2 cancel? $\endgroup$ – shrub Aug 26 '18 at 14:23

By rewriting

$$\eqalign{ l(\lambda_1, \lambda_2) &= y_1\log(\lambda_1F_1) - \lambda_1 F_1 - \log(y_1!)+y_2\log(\lambda_2F_2)-\lambda_2F_2-\log(y_2!) \\ &= y_1\left(\log(\lambda_1)+\log(F_1)\right) - \lambda_1 F_1 - \log(y_1!) \\&\quad+ y_2\left(\log(\lambda_2)+\log(F_2)\right)-\lambda_2F_2-\log(y_2!) \\ &= y_1\log(\lambda_1) - \lambda_1 F_1 + y_2\log(\lambda_2) - \lambda_2 F_2 \\ &\quad+\left[y_2\log(F_2) - \log(y_2!) + y_2 \log(F_2) - \log(y_2!)\right] }$$

you can see that

$$l(\lambda_1,\lambda_2) = y_1\log(\lambda_1) - \lambda_1 F_1 + y_2\log(\lambda_2) - \lambda_2 F_2 + C$$

where $C = y_2\log(F_2) - \log(y_2!) + y_2 \log(F_2) - \log(y_2!)$ does not depend on $(\lambda_1,\lambda_2).$

In general, if the answer is not obvious (and in many situations it is obvious), you can identify the form of $l$ up to an additive constant by means of differentiation and re-integration. This approach is mindless, requiring no insight or cleverness, but only a mechanical grasp of (univariate) Calculus--whence it can usually be carried out with symbolic mathematical software like Mathematica.

Abstractly, suppose $f$ is a differentiable function of a variable $x$ plus stuff that does not depend on $x$:

$$f(x) = g(x) + C.$$

Then because the derivative of any constant is zero, differentiation removes $C$:

$$f^\prime(x) = g^\prime(x).$$

Integrate (in any fashion you like) to obtain an answer. (If you're doing this manually, you don't have to be good with integrals: the original form of $l$ gives you a strong hint concerning what the solution is likely to be. Guess-and-check often works well.)

When your function, such as $l,$ depends on several variables, apply this procedure in turn to each of those variables.

For example, doing this for your particular $l$ (by means of the Sum, Product, and Chain rules of differentiation) gives

$$\frac{\partial l(\lambda_1,\lambda_2)}{\partial\lambda_1} = \frac{y_1}{\lambda_1} - F_1.$$

The indefinite integral is

$$l(\lambda_1,\lambda_2) = \int \left(\frac{y_1}{\lambda_1} - F_1\right)d\lambda_1 = y_1\log(\lambda_1) - \lambda_1 F_2 + C_1\tag{1}$$

for a value $C_1$ which does not depend on $\lambda_1.$ Similarly, differentiating with respect to $\lambda_2$ you will obtain

$$l(\lambda_1,\lambda_2) = \int \left(\frac{y_2}{\lambda_2} - F_2\right)d\lambda_2 = y_2\log(\lambda_2) - \lambda_2 F_2 + C_2\tag{2}$$

where now $C_2$ does not depend on $\lambda_2.$ Equating $(1)$ and $(2)$ shows that the $\lambda_2$-dependence of $l$ must be expressed entirely in $C_1$ and $y_2\log(\lambda_2)-\lambda_2F_2,$ with a similar conclusion for the $\lambda_1$ dependence. The only way this can occur is for

$$l(\lambda_1,\lambda_2) = y_1\log(\lambda_1) - \lambda_1 F_1 + y_2\log(\lambda_2) - \lambda_2 F_2 + C$$

where now $C$ does not depend on either of $\lambda_1$ or $\lambda_2.$ Sometimes you will obtain a different looking answer using this method, but (provided the calculations are correct) it will still be a correct answer.


$y_1 \log(\lambda_1 F_1) = y_1 \log(\lambda_1) + y_1 \log(F1)$ and the latter term does not depend on the variables that you want to optimize in. This is not to be read like “the second expression is equal to the first” i.e. in that sense it is a really bad idea to call both $l$ because they are different things! However, every optimal point for the first expression is an optimal point for the second and vice versa.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.