Regularity conditions hint

Consider a well-defined function $$\psi(x,\theta)$$. Assume that it is a smooth function, differentiable, has a finite expectation and a finite second moment. $$E\dfrac{\partial \psi(X,\theta)}{\partial \theta} \neq 0$$ and is finite as well. The text book then provides the following hint: $$E \sup \biggl({\dfrac{1}{n}\sum_{i=1}^{n} \left|\dfrac{\partial\psi(X_i,\theta)}{\partial\theta} - \dfrac{\partial\psi(X_i,\theta(P))}{\partial\theta(P)}\right|:|\theta-\theta(P)|\leq \epsilon_n}\biggr) \leq E \sup \biggl({\left|\dfrac{\partial\psi(X_1,\theta)}{\partial\theta} - \dfrac{\partial\psi(X_1,\theta(P))}{\partial\theta(P)}\right|:|\theta-\theta(P)|\leq \epsilon_n}\biggr)$$

EDIT: Assume $$X_1, \ldots, X_n$$ are iid random variables. The parameter $$\theta(P)$$ is given by the solution of $$\int{\psi(x) f(x,\theta) dx} = 0$$, where as $$\theta$$ is a free variable.

How does the above relation hold? Does it hold all the time?

• You use $\theta$ in two distinct ways, so please explain the distinction between "$\theta$" and "$\theta(P).$" Without any context we can't even guess.
– whuber
Mar 9 at 0:16
• Clarified this a bit. $\theta(P)$ is the minimum contrast estimator whereas $\theta$ is the parameter we are trying to estimate. Mar 9 at 0:26

Written in that way, it seems complicated. But what the hint really says is just the following simple inequality: \begin{align} \sup_{\theta \in \Theta}(f(\theta) + g(\theta)) \leq \sup_{\theta \in \Theta}f(\theta) + \sup_{\theta \in \Theta}g(\theta), \end{align} for any real functions $$f, g$$ and non-empty set $$\Theta$$.
Now use the linearity of expectation and the i.i.d. assumption of $$X_1, \ldots, X_n$$.
• Added a relevant post at Math.SE. We use this quite often but I wonder whether it has any name to be used as reference. Or just the generic name "$\sup$ inequality". Mar 9 at 2:51