2
$\begingroup$

Assume that $f ({\bf x}; \theta): \mathbb{R}^p \times \Theta \to \mathbb{R}$, where ${\bf x}$ is the vector of inputs (with some distribution) and $\theta$ is the vector of parameters.

Also, assume that $E_{\bf x}[f({\bf x};\theta)]$ is maximized at $\theta^*$.

To estimate $\theta^*$ I solve the following problem: $$ \widehat{\theta} = \arg\max_{\theta} \frac{1}{n} \sum_{i=1}^n f ({\bf x}_i; \theta), \qquad (1) $$ where ${\bf x}_i$ are i.i.d. samples.

If $f$ is differentiable, under some regularity conditions, it is very well-known that $\widehat{\theta}$ is a consistent estimator of $\theta^*$, and is also asymptotically Normal.

In my research I have encountered a problem where $f$ is discontinuous and non-differentiable (actually it is piece-wise constant), but its expected value $E_{\bf x}[f({\bf x};\theta)]$ is continuous and differentiable.

I have shown that solving (1) gives a consistent estimator for $\theta^*$, but cannot prove the asymptotic normality of the estimator.

To me this seems like a general problem which must have been addressed in the literature, but cannot find any reference.

Is there a paper that has addressed this problem?

| cite | improve this question | | | | |
$\endgroup$
  • $\begingroup$ $f(x_i; \theta)$ is piece-wise constant as a function of $\theta$? This is not a typical problem and in fact there are plenty of cases where this won't even lead to a consistent estimator of $\theta$. $\endgroup$ – Cliff AB Jan 11 '16 at 21:22
  • $\begingroup$ Given that you do have a consistent estimator, I would assume that you have something like the fact that the "grid" of $f$ gets finer as $x \rightarrow \infty$? $\endgroup$ – Cliff AB Jan 11 '16 at 21:23
  • $\begingroup$ Yes, that's true. Something like the inner product of ${\bf x}$ and $\theta$ identifies the changing points and as ${\bf x} \to \infty$, smaller changes in $\theta$ result in more changes of the function (i.e., "finer grid"). $\endgroup$ – MMM Jan 11 '16 at 21:30
  • 3
    $\begingroup$ I'm actually going to doubt the asymptotic normality of the estimator. I have done work on a problem with a similar characteristic (NPMLE for interval censored data) and it does not have asymptotic normality. In fact, it does not have an $n^{1/2}$ convergence rate, but rather a $n^{1/3}$ in a special case and an unknown convergence rate in the more general case. Not an easy problem! $\endgroup$ – Cliff AB Jan 11 '16 at 21:37
0
$\begingroup$

Asymptotic normality results with non smooth objective functions has been addressed in the Econometric literature by Daniels (1961), Huber (1967), Pollard (1985), and Pakes and Pollard (1989).

Theorem.

Suppose that $\hat{Q}_n(\hat{\theta}) \ge sup_{\theta\in\Theta}\hat{Q}_n(\theta) - o_p(n^{-1}), \hat{\theta}\to\theta_0$, and (i) $Q_o(\theta)$ is maximized on $\Theta$ at $\theta_o$; (ii) $\theta_0$ is an interior point of $\Theta$, (iii) $Q_o(\theta)$ is twice differentiable at $\theta_0$ with nonsingular second derivative $H$; (iv) $\sqrt{n}\hat{D}\to_d N(0,\Omega)$; (v) for any $\delta_n\to0$, $sup_{||\theta-\theta_n|| \le \delta_n}|\hat{R}_n(\theta)/[1+\sqrt{n}||\theta-\theta_0||]\to_p 0$. Then $\sqrt{n}(\theta-\theta_0)\to_d N(0,H^{-1}\Omega H^{-1})$.

Note: For reference you can refer to chapter on Large Sample Estimation and Hypothesis Testing by Newey and McFadden (Handbook of Econometrics).

| cite | improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.