1
$\begingroup$

I have the following setting: $$ x_k \sim N(\mu,\sigma^2 + \hat{\delta}^2_k),k=1,\dots,K, $$

where $\{x_k,k=1,\dots,K\}$ - observed data, $\{\hat{\delta}^2_k\,k=1,\dots,K \}$ are known parameters (just consider them fixed), while $(\mu,\sigma^2)$ are unknown. My primary goal - making inference about $\mu$. Does anyone know of any results for this particular case in terms of consistency and asymptotic behavior of MLE estimator for $\mu$ as $K\rightarrow \infty$?

For the time being I am simply using a $\chi^2$ approximation for the likelihood ratio test, but I don't have solid theoretical argument for that as the data is not identically distributed. It also complicates things that I fail to get a closed-form solution for $\hat{\mu}_{mle}$

I have tried Hoadley's paper on "Asymptotic Properties of MLE for Independent Non-Identically distributed case" http://projecteuclid.org/download/pdf_1/euclid.aoms/1177693066, but verifying their general conditions is not coming to me easy as of yet.

Please let me know if anyone encountered this particular kind of a problem and knows what regularity conditions are needed for nice classic MLE properties. Obviously there should be a certain upper bound on values $\{ \hat{\delta}^2_k,\ k=1,\dots, \}$, but what kind of bound, and even with that bound - how to prove consistency/asymptotic normality.

|cite|improve this question
$\endgroup$
  • $\begingroup$ The "hat" over your $\delta$s suggests they're estimated from data rather than actually known. Is it the case that you really know the true values rather than estimates of them? $\endgroup$ – Glen_b -Reinstate Monica May 11 '17 at 23:34
  • $\begingroup$ You can just consider them fixed/known, yes. The "hat" is more like a notion for them to be distinguished from the usual notation for unknown parameters. $\endgroup$ – UsDAnDreS May 11 '17 at 23:57
  • $\begingroup$ ...In the meantime you may want to have a look at stats.stackexchange.com/a/94128/28746 $\endgroup$ – Alecos Papadopoulos May 20 '17 at 16:24
1
$\begingroup$

You can find much more accessible conditions for consistency and asymptotic normality of MLE in Hayashi's Econometrics, ch. 7.,in the general context of Extremum Estimators and its sub-class, the M-estimators. Hayashi has also references for detailed proofs on the conditions.

The MLE with independent observations belongs to this subclass, because it maximizes a "sample average", an average of a real-valued function of the data and the unknown parameters (note that with independent observations the log-likelihood of the sample is certainly a sum, and we can divide it by the sample size without affecting the solution).

So (in general notation)

$$\hat \theta_{MLE} = \text{argmax}_{\theta} \left\{\frac 1n \sum_{i=1}^n \ell_i(x_i;\theta)\right\}$$

where $\ell_i$ is the log-likelihood of observation $i$.

For consistency, there are two-three alternative sets of conditions. Common to all conditions are:
1) The parameters lie in the interior of the parameter space

2) $\ell_i(x_i;\theta)$ is measurable (if it is continuous, it is measurable)

3) The objective function $\frac 1n \sum_{i=1}^n \ell_i(x_i;\theta)$ converges in probability to some function, say $\ell_0(\theta)$

4) $\ell_0(\theta)$ is uniquely maximized at the true parameter vector (say $\theta_0$)

Then moreover:
1st Alternative : if the parameter space is compact, and convergence is uniform, we obtain consistency.

2nd Alternative : if the parameter space is not compact, then if the log-likelihood is concave and convergence is just pointwise, we again obtain consistency.

I ' ll leave asymptotic normality for the OP to look up and explore.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ I can't find these conditions on the given reference of Hayashi's econometrics chapter 7, it seems to me Hayashi always assumes identically distributed random variables. where could I find this theorem you cite? $\endgroup$ – Monolite Apr 1 '19 at 15:26
  • $\begingroup$ @Monolite Are you sure? See the various theorems in Hayashi pp. 456-465. $\endgroup$ – Alecos Papadopoulos Apr 1 '19 at 18:33
  • $\begingroup$ It seems to me that hayashi's notation $Q_n(\theta) := \frac{1}{n} \sum_{t=1}^n \log f(y_t | x_t ; \theta)$ does not assume the $f$ to vary as in your formula $\frac 1n \sum_{i=1}^n \ell_i(x_i;\theta)$ where $\ell_i$ explicitly depends on $i$ . $\endgroup$ – Monolite Apr 1 '19 at 18:46
  • 1
    $\begingroup$ @Monolite You are right that Hayashi passes without notice form general data to i.i.d. data. A paper exactly on the case of "independent non-identical" is Bruce Hoadley (1971) "Asymptotic Properties of Maximum Likelihood Estimators for the Independent Not Identically Distributed Case". For non-standard conditions in general, see also stats.stackexchange.com/a/68866/28746 $\endgroup$ – Alecos Papadopoulos Apr 1 '19 at 18:59
  • 1
    $\begingroup$ @Monolite Yes, it was distilling what one can find in Hayashi, Amemiya, etc $\endgroup$ – Alecos Papadopoulos Apr 1 '19 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.