Maximum Likelihood of a sum of products - any way to transform? I have a likelihood function in this form:
$$L(\lambda) = \sum_i \prod_j f(i,j, \lambda)$$
$f(i,j, \lambda)$ are probabilities, so that their product gets very small. If I try and calculate it in python, it's giving wrong answers due to machine errors. 
If it wasn't for the summand I would take the log-likelihood, but I'm not sure what to do. Is there a way to transform this likelihood to make it easier to compute?
 A: Here's a way to compute the log likelihood, avoiding numerical under/overflow by working with log probabilities and using the log-sum-exp trick.
Let $p_i(\lambda)$ be the product of the probabilities over $j$, given $i$ and $\lambda$:
$$p_i(\lambda) = \prod_j f(i,j,\lambda)$$
Obviously, the log can be computed as:
$$\log p_i(\lambda) = \sum_j \log f(i,j,\lambda)$$
Unlike computing the product of many small values, computing the sum of their logs has little danger of underflow. Now, let $v^*(\lambda)$ be the maximal value of $\log p_i(\lambda)$ over all $i$:
$$v^*_\lambda = \max_i \log p_i(\lambda)$$
Using the log-sum-exp trick, the log likelihood can be computed as:
$$\log L(\lambda) =
v^*(\lambda)
+ \log \sum_i \exp \Big(
  \log p_i(\lambda)
  - v^*(\lambda)
\Big)$$
Subtracting $v^*(\lambda)$ inside the exponential terms has the effect of bringing all values into a small range near zero, reducing the risk of under/overflow when we exponentiate. Many numerical computing libraries have a logsumexp() function for handling this computation in a numerically safe way (e.g. see here for python).
