# Asymptotic normality for nonsmooth objective functions

Assume that $f ({\bf x}; \theta): \mathbb{R}^p \times \Theta \to \mathbb{R}$, where ${\bf x}$ is the vector of inputs (with some distribution) and $\theta$ is the vector of parameters.

Also, assume that $E_{\bf x}[f({\bf x};\theta)]$ is maximized at $\theta^*$.

To estimate $\theta^*$ I solve the following problem: $$\widehat{\theta} = \arg\max_{\theta} \frac{1}{n} \sum_{i=1}^n f ({\bf x}_i; \theta), \qquad (1)$$ where ${\bf x}_i$ are i.i.d. samples.

If $f$ is differentiable, under some regularity conditions, it is very well-known that $\widehat{\theta}$ is a consistent estimator of $\theta^*$, and is also asymptotically Normal.

In my research I have encountered a problem where $f$ is discontinuous and non-differentiable (actually it is piece-wise constant), but its expected value $E_{\bf x}[f({\bf x};\theta)]$ is continuous and differentiable.

I have shown that solving (1) gives a consistent estimator for $\theta^*$, but cannot prove the asymptotic normality of the estimator.

To me this seems like a general problem which must have been addressed in the literature, but cannot find any reference.

Is there a paper that has addressed this problem?

• $f(x_i; \theta)$ is piece-wise constant as a function of $\theta$? This is not a typical problem and in fact there are plenty of cases where this won't even lead to a consistent estimator of $\theta$. – Cliff AB Jan 11 '16 at 21:22
• Given that you do have a consistent estimator, I would assume that you have something like the fact that the "grid" of $f$ gets finer as $x \rightarrow \infty$? – Cliff AB Jan 11 '16 at 21:23
• Yes, that's true. Something like the inner product of ${\bf x}$ and $\theta$ identifies the changing points and as ${\bf x} \to \infty$, smaller changes in $\theta$ result in more changes of the function (i.e., "finer grid"). – MMM Jan 11 '16 at 21:30
• I'm actually going to doubt the asymptotic normality of the estimator. I have done work on a problem with a similar characteristic (NPMLE for interval censored data) and it does not have asymptotic normality. In fact, it does not have an $n^{1/2}$ convergence rate, but rather a $n^{1/3}$ in a special case and an unknown convergence rate in the more general case. Not an easy problem! – Cliff AB Jan 11 '16 at 21:37

Suppose that $\hat{Q}_n(\hat{\theta}) \ge sup_{\theta\in\Theta}\hat{Q}_n(\theta) - o_p(n^{-1}), \hat{\theta}\to\theta_0$, and (i) $Q_o(\theta)$ is maximized on $\Theta$ at $\theta_o$; (ii) $\theta_0$ is an interior point of $\Theta$, (iii) $Q_o(\theta)$ is twice differentiable at $\theta_0$ with nonsingular second derivative $H$; (iv) $\sqrt{n}\hat{D}\to_d N(0,\Omega)$; (v) for any $\delta_n\to0$, $sup_{||\theta-\theta_n|| \le \delta_n}|\hat{R}_n(\theta)/[1+\sqrt{n}||\theta-\theta_0||]\to_p 0$. Then $\sqrt{n}(\theta-\theta_0)\to_d N(0,H^{-1}\Omega H^{-1})$.