Maximum likelihood estimation when the model is misspecified (and the true data generating process is a mixture model)

Question

I'm interested in the properties of maximum likelihood estimators under a particular form of model misspecification:

• We observe data $\left\{X_i\right\}$ generated from a finite mixture model
• Let $\pi_k$ denote the mixture weight for component $k$, and let $\theta_k$ denote its parameters
• All mixture components are from the same family of probability distributions $F\left(x; \theta\right)$

The true data generating process is $H(x) = \sum_k \pi_k \, F\left(x; \theta_k \right)$. I use $H$ instead of $F$ because the distributions may belong to different families: for example, if $F$ is Gaussian, $H$ (which is a mixture of Gaussians) will in general be non-Gaussian.

I am interested in the case where certain elements of the parameters $\theta_k$ are constant across components, and do not vary with $k$. We can write $\theta_k \equiv \left(\theta^\star, \gamma_k\right)$ where

• $\theta^\star$ is the part of the parameter vector that is constant across mixture components
• $\gamma_k$ is the part of the parameter vector that varies with $k$

Suppose we run maximum likelihood estimation using data $X_i \sim H(x)$. However, instead of correctly specifying and estimating a mixture model, we maximize the likelihood for the model $F\left(x; \theta \right)$.

What are the properties of this misspecified estimator? In particular, will we have a consistent estimator of $\theta^\star$, the part of the parameter vector that is constant/identical across mixture components?

I'm interested in references that address this question.

---

Here is a specific example (although I am interested in the general case described above):

• $H$ is a mixture of two Gaussians
• Component 1 has weight $\pi_1 = 0.5$ and is $\mathcal{N}(\mu, \sigma^2_1)$
• Component 2 has weight $\pi_2 = 0.5$ and is $\mathcal{N}(\mu, \sigma^2_2)$
• The mean $\mu$ is the same for the two components, but the variances $\sigma^2_1$ and $\sigma^2_2$ differ. For example, we might have $\mu = 10$, $\sigma^2_1 = 1$, $\sigma^2_2 = 5$.

If we estimate $\theta = \left(\mu, \sigma^2\right)$ by maximum likelihood (using a misspecified model that assumes the data is $\mathcal{N}(\mu, \sigma^2)$), would we have a consistent estimator of $\mu$?

In R:

library(mixtools)

n_obs <- 1000

## Mu is the same for both components
x <- rnormmix(n_obs, lambda=c(0.5, 0.5), mu=10, sigma=sqrt(c(1, 5)))

hist(x)

negative_log_likelihood <- function(theta) {
    ## The likelihood is misspecified (it isn't a mixture model)
    return(-sum(dnorm(x, mean=theta[1], sd=exp(theta[2]), log=TRUE)))
}

## We can't possibly have a consistent estimator of sigma 
## (because it varies across components),
## but do we have a consistent estimator of mu?
optim_result <- optim(par=c(0, 3), fn=negative_log_likelihood)

mu_hat <- optim_result$par[1]
sigma_hat <- exp(optim_result$par[2])