Asymptotic normality for nonsmooth objective functions

Question

Assume that $f ({\bf x}; \theta): \mathbb{R}^p \times \Theta \to \mathbb{R}$, where ${\bf x}$ is the vector of inputs (with some distribution) and $\theta$ is the vector of parameters.

Also, assume that $E_{\bf x}[f({\bf x};\theta)]$ is maximized at $\theta^*$.

To estimate $\theta^*$ I solve the following problem: $$ \widehat{\theta} = \arg\max_{\theta} \frac{1}{n} \sum_{i=1}^n f ({\bf x}_i; \theta), \qquad (1) $$ where ${\bf x}_i$ are i.i.d. samples.

If $f$ is differentiable, under some regularity conditions, it is very well-known that $\widehat{\theta}$ is a consistent estimator of $\theta^*$, and is also asymptotically Normal.

In my research I have encountered a problem where $f$ is discontinuous and non-differentiable (actually it is piece-wise constant), but its expected value $E_{\bf x}[f({\bf x};\theta)]$ is continuous and differentiable.

I have shown that solving (1) gives a consistent estimator for $\theta^*$, but cannot prove the asymptotic normality of the estimator.

To me this seems like a general problem which must have been addressed in the literature, but cannot find any reference.

Is there a paper that has addressed this problem?

Answer 1

Asymptotic normality results with non smooth objective functions has been addressed in the Econometric literature by Daniels (1961), Huber (1967), Pollard (1985), and Pakes and Pollard (1989).

Theorem.

Suppose that $\hat{Q}_n(\hat{\theta}) \ge sup_{\theta\in\Theta}\hat{Q}_n(\theta) - o_p(n^{-1}), \hat{\theta}\to\theta_0$, and (i) $Q_o(\theta)$ is maximized on $\Theta$ at $\theta_o$; (ii) $\theta_0$ is an interior point of $\Theta$, (iii) $Q_o(\theta)$ is twice differentiable at $\theta_0$ with nonsingular second derivative $H$; (iv) $\sqrt{n}\hat{D}\to_d N(0,\Omega)$; (v) for any $\delta_n\to0$, $sup_{||\theta-\theta_n|| \le \delta_n}|\hat{R}_n(\theta)/[1+\sqrt{n}||\theta-\theta_0||]\to_p 0$. Then $\sqrt{n}(\theta-\theta_0)\to_d N(0,H^{-1}\Omega H^{-1})$.

Note: For reference you can refer to chapter on Large Sample Estimation and Hypothesis Testing by Newey and McFadden (Handbook of Econometrics).