Setup

I'm reading (Xu and Knight 2010), which is a paper about estimating finite Gaussian mixture models using the CECF (Continuous Empirical Characteristic Function) method. The basic idea is to minimize a weighted distance between the empirical characteristic function (ECF) and the theoretical one (CF) with respect to parameters of the mixture distribution.

The weighted distance is given by this integral:

$$D(\theta; \pmb{r}) = \int_{-\infty}^{\infty} |\mathrm{ECF}(t; \pmb{r}) - \mathrm{CF}(t, \theta)|^2 e^{-bt^2} dt$$

Here, $$\pmb{r}$$ is the vector of data used to compute the empirical characteristic function.

The point of the paper is that this integral can be solved analytically, and the distance $$D(\theta; \pmb{r})$$ becomes a function that one can compute without integrals. I'll have to put a screenshot here since the function is pretty long:

This function involves the weighting parameter $$b$$ that influences the asymptotic covariance of the estimate. So I think the distance should really be written $$D(\theta; \pmb{r}, b)$$.

Asymptotic covariance

By Proposition 2, the estimated mixture parameters $$\hat\theta = \arg\min_{\theta} D(\theta; \pmb{r}, b)$$ for a fixed $$b$$ are asymptotically normal with the following covariance matrix:

$$\begin{cases} \mathrm{Var}(\hat\theta) &= \Lambda^{-1}\Omega\Lambda^{-1}\\ \Lambda_{ij} &= \mathbb{E} \dfrac{ \partial D^2(\theta; \pmb{r}, b) }{ \partial \theta_i \partial \theta_j }\\ \Omega_{ij} &= \mathbb{E} \left( \dfrac{\partial D(\theta; \pmb{r}, b)}{\partial \theta_i} \dfrac{\partial D(\theta; \pmb{r}, b)}{\partial \theta_j} \right) \end{cases}$$

Question

How does one compute these expectations?

To me this looks similar to the Fisher information matrix which one estimates using the Hessian of the log-likelihood at the estimate $$\hat\theta$$. This makes sense because the log-likelihood is a sum, so when one differentiates it, they get a sum of derivatives (or Hessians), and this sum is basically the average ("empirical expectation").

But here $$D(\theta; \pmb{r}, b)$$ isn't just a sum over the data $$\pmb r$$. In fact, the first quantity is this double nested sum over the data, and the last quantity involves $$K$$ sums over the data. So, the result doesn't look like the average of anything to me.

Am I overthinking this, and is $$\Lambda$$ just the Hessian of $$D$$ evaluated at the estimates $$\hat\theta$$?

How do I compute $$\Omega_{ij}$$? I was thinking about computing the derivatives $$\partial D(\theta; r_n, b) / \partial\theta_i$$ for each data point $$r_n \in \pmb r$$, then calculating this sum:

$$\Omega_{ij} \approx \frac1N \sum_{n=1}^N \left[ \frac{ \partial D(\theta;r_n,b) }{\partial\theta_i} \frac{ \partial D(\theta;r_n,b) }{\partial\theta_j} \right]$$

Is this the right approach?

Proof of Lemma 1 in the Appendix says:

Recall that $$\Omega_{ij} = \int \left(\frac{\partial D(\theta;r)}{\partial\theta_i} \frac{\partial D(\theta;r)}{\partial\theta_j}\right) \exp(-bt^2) dt$$

...which seems weird because $$D(\theta;r)$$ doesn't depend on $$t$$, so one can move the product of derivatives out of the integral and compute $$\left(\frac{\partial D(\theta;r)}{\partial\theta_i} \frac{\partial D(\theta;r)}{\partial\theta_j}\right) \int \exp(-bt^2) dt$$, but this isn't an expectation, is it?