Why nonparametric maximum likelihood of mixture is convex Consider $x_i \sim N(\mu_i, 1)$ where $i = 1, \ldots, n$ and assume $\mu_i$ is generated i.i.d. from an unknown distribution $F$. We are interested in estimating the unknown $\mu_i$. One way to solve the problem is by nonparametric maximum likelihood as follows.
$$\min_{F\in \mathcal{F}} - \sum_{i=1}^n log \int \phi(x-\mu)dF(\mu)$$where $\phi$ denotes a standard normal distribution and $\mathcal{F}$ is a convex set.
As claimed in many places without explaination (ref1, ref2, ref3), this is a convex optimization problem. Why? The integrand is a mixture and it is not clear to me how this becomes convex. Did I miss something?
 A: A functional $J : \mathcal{F} \to \mathbb{R}$ is convex if the space $\mathcal{F}$ is convex and if $J(tf + (1 - t)g) \le tJ(f) + (1 - t)J(g)$ for all $t \in [0, 1]$ and all functions $f, g \in \mathcal{F}$. For notational simplicity, let $L_x$ denote the functional $$L_x (F) = \int \phi(x - \mu) dF (\mu)
$$ and let the functional of interest be denoted by
$$J(F) = \sum_{i = 1}^n \left( -\log \int \phi(x_i - \mu ) dF(\mu) \right) = \sum_{i = 1}^n  (-\log L_{x_i} (F) ).$$
Here we have a sum of a convex and non-decreasing function ($-\log$) of a linear functional ($L_{x_i}$), which together make $J$ a convex functional. The functional $L_{x_i}$ is linear because the Riemann-Stieltjes integral $\int f(x) dg(x) $ is linear in $g$, i.e. that $\int f(x) d(g + h)(x) = \int f(x) dg(x) + \int f(x) dh(x)$ (See these notes for proof).
To show that $J$ is convex, start by noting that $-\log \circ L$ is convex, because
\begin{align}
-\log (L_{x_i} (tF + (1 - t)G)) &= -\log (t L_{x_i}(F) + (1 - t)L_{x_i}(G)) \quad \text{ (by linearity)} \\
&\le t(-\log L_{x_i}(F)) + (1 - t)(-\log(L_{x_i}(G)) \quad \text{ (by Jensen's Inequality)} \\
\end{align}
Since this inequality holds for all $x_i$, sum up the $n$ inequalities you get from each of the $L_{x_i}$ to get
$$
J(tF + (1 - t)G) \le t \sum_{i = 1}^n (-\log (L_{x_i} (F))) + (1 - t) \sum_{i = 1}^n (-\log (L_{x_i} (G))) = t J(F) + (1 - t) J(G)
$$
which proves that $J$ is convex.
