Regularity conditions hint

Question

Consider a well-defined function $\psi(x,\theta)$. Assume that it is a smooth function, differentiable, has a finite expectation and a finite second moment. $E\dfrac{\partial \psi(X,\theta)}{\partial \theta} \neq 0$ and is finite as well. The text book then provides the following hint: $E \sup \biggl({\dfrac{1}{n}\sum_{i=1}^{n} \left|\dfrac{\partial\psi(X_i,\theta)}{\partial\theta} - \dfrac{\partial\psi(X_i,\theta(P))}{\partial\theta(P)}\right|:|\theta-\theta(P)|\leq \epsilon_n}\biggr) \leq E \sup \biggl({\left|\dfrac{\partial\psi(X_1,\theta)}{\partial\theta} - \dfrac{\partial\psi(X_1,\theta(P))}{\partial\theta(P)}\right|:|\theta-\theta(P)|\leq \epsilon_n}\biggr)$

EDIT: Assume $X_1, \ldots, X_n$ are iid random variables. The parameter $\theta(P)$ is given by the solution of $\int{\psi(x) f(x,\theta) dx} = 0$, where as $\theta$ is a free variable.

How does the above relation hold? Does it hold all the time?

Answer 1

Written in that way, it seems complicated. But what the hint really says is just the following simple inequality: \begin{align} \sup_{\theta \in \Theta}(f(\theta) + g(\theta)) \leq \sup_{\theta \in \Theta}f(\theta) + \sup_{\theta \in \Theta}g(\theta), \end{align} for any real functions $f, g$ and non-empty set $\Theta$.

Now use the linearity of expectation and the i.i.d. assumption of $X_1, \ldots, X_n$.