I've got a simple question about deriving log-likelihoods. I am stumped by the following-->

If the log-likelihood is: 𝑙(πœ†1,πœ†2) = 𝑦1 log(πœ†1𝐹1)βˆ’πœ†1𝐹1 βˆ’log((𝑦1)!)+𝑦2 log(πœ†2𝐹2) βˆ’πœ†2𝐹2 βˆ’log⁑ ((𝑦2)!)

Then if we omit terms not depending on πœ†π‘– the log-likelihood becomes: 𝑙(πœ†1,πœ†2) = 𝑦1 log(πœ†1)βˆ’πœ†1𝐹1 +𝑦2 log(πœ†2)βˆ’πœ†2𝐹2

I am confused why we're able to drop the F1 and F2 from the log expressions? Aren't they dependent on πœ†π‘–? Am I just forgetting a simple log law...?

  • $\begingroup$ You do: log(ab)=log(a)+log(b)... $\endgroup$ – Fabian Werner Aug 26 '18 at 12:52
  • $\begingroup$ Thank you, though I'm still lost. Why do F1 and F2 cancel? $\endgroup$ – shrub Aug 26 '18 at 14:23

By rewriting

$$\eqalign{ l(\lambda_1, \lambda_2) &= y_1\log(\lambda_1F_1) - \lambda_1 F_1 - \log(y_1!)+y_2\log(\lambda_2F_2)-\lambda_2F_2-\log(y_2!) \\ &= y_1\left(\log(\lambda_1)+\log(F_1)\right) - \lambda_1 F_1 - \log(y_1!) \\&\quad+ y_2\left(\log(\lambda_2)+\log(F_2)\right)-\lambda_2F_2-\log(y_2!) \\ &= y_1\log(\lambda_1) - \lambda_1 F_1 + y_2\log(\lambda_2) - \lambda_2 F_2 \\ &\quad+\left[y_2\log(F_2) - \log(y_2!) + y_2 \log(F_2) - \log(y_2!)\right] }$$

you can see that

$$l(\lambda_1,\lambda_2) = y_1\log(\lambda_1) - \lambda_1 F_1 + y_2\log(\lambda_2) - \lambda_2 F_2 + C$$

where $C = y_2\log(F_2) - \log(y_2!) + y_2 \log(F_2) - \log(y_2!)$ does not depend on $(\lambda_1,\lambda_2).$

In general, if the answer is not obvious (and in many situations it is obvious), you can identify the form of $l$ up to an additive constant by means of differentiation and re-integration. This approach is mindless, requiring no insight or cleverness, but only a mechanical grasp of (univariate) Calculus--whence it can usually be carried out with symbolic mathematical software like Mathematica.

Abstractly, suppose $f$ is a differentiable function of a variable $x$ plus stuff that does not depend on $x$:

$$f(x) = g(x) + C.$$

Then because the derivative of any constant is zero, differentiation removes $C$:

$$f^\prime(x) = g^\prime(x).$$

Integrate (in any fashion you like) to obtain an answer. (If you're doing this manually, you don't have to be good with integrals: the original form of $l$ gives you a strong hint concerning what the solution is likely to be. Guess-and-check often works well.)

When your function, such as $l,$ depends on several variables, apply this procedure in turn to each of those variables.

For example, doing this for your particular $l$ (by means of the Sum, Product, and Chain rules of differentiation) gives

$$\frac{\partial l(\lambda_1,\lambda_2)}{\partial\lambda_1} = \frac{y_1}{\lambda_1} - F_1.$$

The indefinite integral is

$$l(\lambda_1,\lambda_2) = \int \left(\frac{y_1}{\lambda_1} - F_1\right)d\lambda_1 = y_1\log(\lambda_1) - \lambda_1 F_2 + C_1\tag{1}$$

for a value $C_1$ which does not depend on $\lambda_1.$ Similarly, differentiating with respect to $\lambda_2$ you will obtain

$$l(\lambda_1,\lambda_2) = \int \left(\frac{y_2}{\lambda_2} - F_2\right)d\lambda_2 = y_2\log(\lambda_2) - \lambda_2 F_2 + C_2\tag{2}$$

where now $C_2$ does not depend on $\lambda_2.$ Equating $(1)$ and $(2)$ shows that the $\lambda_2$-dependence of $l$ must be expressed entirely in $C_1$ and $y_2\log(\lambda_2)-\lambda_2F_2,$ with a similar conclusion for the $\lambda_1$ dependence. The only way this can occur is for

$$l(\lambda_1,\lambda_2) = y_1\log(\lambda_1) - \lambda_1 F_1 + y_2\log(\lambda_2) - \lambda_2 F_2 + C$$

where now $C$ does not depend on either of $\lambda_1$ or $\lambda_2.$ Sometimes you will obtain a different looking answer using this method, but (provided the calculations are correct) it will still be a correct answer.


$y_1 \log(\lambda_1 F_1) = y_1 \log(\lambda_1) + y_1 \log(F1)$ and the latter term does not depend on the variables that you want to optimize in. This is not to be read like β€œthe second expression is equal to the first” i.e. in that sense it is a really bad idea to call both $l$ because they are different things! However, every optimal point for the first expression is an optimal point for the second and vice versa.


Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.