# What can go wrong with MLE if I substitute some first-stage estimates instead of some parameters?

Suppose that initially I am dealing with the log-likelihood function $\log L(\theta_1, \ldots, \theta_m, \theta_{m+1}, \ldots, \theta_k)$, where $\theta_j \in \mathbb{R}$.

Suppose that for whatever reason I decided to input into $\log L$ some first-stage estimates $\tilde{\theta}_{m+1}$, $\ldots$, $\tilde{\theta}_k$ obtained in some other way and then maximize $\log L$ over $\theta_1$, $\ldots$, $\theta_m$. All $\tilde{\theta}_{m+1}$, $\ldots$, $\tilde{\theta}_k$ are consistent estimators of the true parameter values $\theta_{0,m+1}$, $\ldots$, $\theta_{0,k}$.

My question is: what can possibly go wrong with the MLE in this case? Does the MLE estimator $\hat{\theta}_1$, $\ldots$, $\hat{\theta}_m$ have the same asymptotic properties as before? Does anything depend on the rates of convergence of $\tilde{\theta}_{m+1}$, $\ldots$, $\tilde{\theta}_k$?

• Good question. I'm working on something that falls into this category. I think this kind of thing is called "plug-in likelihood" but I'm interested in seeing what answers you get. I think this paper might be relevant: sciencedirect.com/science/article/pii/S0304414913000811 Jan 29, 2017 at 21:05

Your technique is essentially maximizing the conditional log-likelihood, conditioned on $\tilde \theta_{m+1},\ldots,\tilde \theta_k$. The complete maximum log-likelihood is the maximum of this conditional maximum across all these other parameters. This is very frequently used to produce likelihood scans, especially when $k=m+1$ and there is only one conditionalized parameter. The maximum log-likelihood as a function of $\tilde \theta_k$ is useful in setting a confidence interval on $\theta_k$.
Philosophically, it is always the case that there are conditional parameters that are fixed -- you could always add extra parameters into your model. Every likelihood function is a conditional likelihood function, and vice versa; the maximization of a conditional log-likelihood function has all the statistical properties you might expect from maximizing a likelihood function. The only differences are non-statistical in nature, dealing with the assumptions behind the maximization. For example, how reasonable is it to simplify the model? Typically you might like to know that you have an exact value for $\tilde \theta_k$, or that there is some domain-specific (non-statistical) argument for it to have a certain value. For example, in OLS (a type of likelihood maximization), it is assumed that the errors are symmetric, gaussian, and independent of the explanatory variables (e.g. non-heteroskedastic). You could always add parameters for skewness, non-gaussianity, and heteroskedasity, but this is frequently judged to be unnecessary.*
In your case, you just have a statistical estimate, with some confidence interval. The critical question is whether your estimates are taken from the same data that is used during the likelihood maximization, or from an independent dataset. In the latter case, you are performing a very common procedure. One ad-hoc procedure you could try to propagate uncertainties from $\tilde \theta$ onto your final result could be to sample your $\tilde \theta$ from within their confidence intervals in a sort of parametric bootstrap, and maximize the conditional log-likelihood for each sample, yielding an expanded confidence interval. Another technique is to let the parameters float in the log-likelihood, but add constraint terms for their confidence intervals; for example, multiplying the likelihood by a gaussian p.d.f. $\exp(-(\theta_k-\tilde \theta_k)^2/2\sigma_k^2)$, ignoring irrelevant constants.
On the other hand, if your estimates $\tilde \theta$ are made with the same data used in the likelihood maximization, yours is a more questionable procedure. Taking the set of $\tilde \theta$ as fixed givens, the conditional log-likelihood maximization is statistically valid, but it is not guaranteed to play nice with whatever confidence intervals you have for your $\tilde \theta$. The above procedures for adding constraint terms to the likelihood or parametrically sampling the parameters are invalid because the parameters are then double-penalized by the same dataset. You could scan through $\tilde \theta_{m+1},\ldots,\tilde \theta_k$, in a grid covering a reasonable confidence interval. Only you can determine if this is better/easier than simply maximizing the entire log-likelihood.