Consider a well-defined function $\psi(x,\theta)$. Assume that it is a smooth function, differentiable, has a finite expectation and a finite second moment. $E\dfrac{\partial \psi(X,\theta)}{\partial \theta} \neq 0$ and is finite as well. The text book then provides the following hint: $E \sup \biggl({\dfrac{1}{n}\sum_{i=1}^{n} \left|\dfrac{\partial\psi(X_i,\theta)}{\partial\theta} - \dfrac{\partial\psi(X_i,\theta(P))}{\partial\theta(P)}\right|:|\theta-\theta(P)|\leq \epsilon_n}\biggr) \leq E \sup \biggl({\left|\dfrac{\partial\psi(X_1,\theta)}{\partial\theta} - \dfrac{\partial\psi(X_1,\theta(P))}{\partial\theta(P)}\right|:|\theta-\theta(P)|\leq \epsilon_n}\biggr)$

EDIT: Assume $X_1, \ldots, X_n$ are iid random variables. The parameter $\theta(P)$ is given by the solution of $\int{\psi(x) f(x,\theta) dx} = 0$, where as $\theta$ is a free variable.

How does the above relation hold? Does it hold all the time?

  • $\begingroup$ You use $\theta$ in two distinct ways, so please explain the distinction between "$\theta$" and "$\theta(P).$" Without any context we can't even guess. $\endgroup$
    – whuber
    Mar 9 at 0:16
  • $\begingroup$ Clarified this a bit. $\theta(P)$ is the minimum contrast estimator whereas $\theta$ is the parameter we are trying to estimate. $\endgroup$
    – redwing21
    Mar 9 at 0:26

1 Answer 1


Written in that way, it seems complicated. But what the hint really says is just the following simple inequality: \begin{align} \sup_{\theta \in \Theta}(f(\theta) + g(\theta)) \leq \sup_{\theta \in \Theta}f(\theta) + \sup_{\theta \in \Theta}g(\theta), \end{align} for any real functions $f, g$ and non-empty set $\Theta$.

Now use the linearity of expectation and the i.i.d. assumption of $X_1, \ldots, X_n$.

  • $\begingroup$ Added a relevant post at Math.SE. We use this quite often but I wonder whether it has any name to be used as reference. Or just the generic name "$\sup$ inequality". $\endgroup$ Mar 9 at 2:51
  • $\begingroup$ @User1865345 Thanks for your edit. It would be helpful. $\endgroup$
    – Zhanxiong
    Mar 9 at 3:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.