Suppose $F(z,\Theta)$ is a continuosly differentiable cumulative density function where $z$ is a random variable defined over $[0,\infty)$ and $\Theta$ is a parameter of the distribution. I have a feeling that the follwoing limit is true but how can something like this be proven in a mathematically acceptable way? (this may not be an easy question and even a reference is appreciated)

$$\lim_{n \rightarrow \infty}\frac{1}{n} \sum_{i=1}^n \frac{\frac{\partial F(z_i, \Theta)}{\partial \Theta }-\frac{\partial F(z_{i-1}, \Theta)}{\partial \Theta } }{{(z_i-z_{i-1})}}\rightarrow \ E\big(\frac{\partial f(z, \Theta)}{\partial \Theta }\big) $$

where $z_i$ is evaluated at $F^{-1}(i/n,\Theta)$ , $i=1,...,n, E, f$ and $F^{-1}$ denote expectation, probability density function (derivative of $F$) and quantile functions.

  • $\begingroup$ I couldn't figure out how to put the equation in the post $\endgroup$ – user41838 Apr 16 '15 at 2:02
  • $\begingroup$ You nearly had it ... please check my edits say what you intended $\endgroup$ – Glen_b Apr 16 '15 at 3:26
  • $\begingroup$ Thanks Glen_b, I have now edited it further. I am not good at LATEX, that "sum" might be better to be replaced with a Sigma sign $\endgroup$ – user41838 Apr 16 '15 at 8:30
  • $\begingroup$ I just put a "\" in front of "sum" for you which does the trick. $\endgroup$ – Glen_b Apr 16 '15 at 9:17
  • $\begingroup$ I believe you need to normalize the LHS by dividing by $n$. Otherwise, the summands each ought to approximate $\partial^2F(z_i,\theta)/{\partial z\partial \theta}$ and the sum will diverge. With the factor of $1/n$ included, this looks like a direct application of the Lebesgue-Stieltjes Mean Value Theorem. $\endgroup$ – whuber Apr 16 '15 at 16:07

The equality is true provided an additional assumption is made.

Let $z_0 = 0$ (as implied, but not stated, in the question). Then, applying the Mean Value Theorem for Lebesgue-Stieltjes integrals of the form $\int g(z) dF(z)$, the right hand side can be analyzed as

$$\eqalign{ \mathbb{E}\left(\frac{\partial f(z,\theta)}{\partial \theta}\right) &= \int_0^\infty\frac{\partial f(z,\theta)}{\partial \theta} dF(z,\theta) \\ &= \sum_{i=1}^n \int_{z_{i-1}}^{z_i}\frac{\partial f(z,\theta)}{\partial \theta} dF(z,\theta) \\ &= \sum_{i=1}^n \left(F(z_i, \theta) - F(z_{i-1}, \theta)\right)\frac{\partial}{\partial\theta}f(z_i^{*},\theta) \\ &= \sum_{i=1}^n \left(\frac{i}{n} - \frac{i-1}{n}\right)\frac{\partial}{\partial\theta}f(z_i^{*},\theta) \\ &= \frac{1}{n}\sum_{i=1}^n \frac{\partial}{\partial\theta}f(z_i^{*},\theta) }$$

where each $z_i^{*} \in [z_{i-1}, z_{i}]$.

Now let's examine the left hand side, once again invoking the MVT (and assuming continuity of $F$, so that mixed partial derivatives can be reordered):

$$\eqalign{ &\frac{1}{z_i - z_{i-1}} \left(\frac{\partial}{\partial\theta}F(z_i,\theta) -\frac{\partial}{\partial\theta}F(z_{i-1},\theta)\right) \\ &= \frac{\partial}{\partial z}\frac{\partial}{\partial\theta} F(z_{i}^{**}, \theta) = \frac{\partial}{\partial \theta}\frac{\partial}{\partial z} F(z_{i}^{**}, \theta) \\ &= \frac{\partial}{\partial \theta}f(z_{i}^{**}, \theta). }$$

where each $z_i^{**} \in [z_{i-1}, z_{i}]$.

Thus the difference between the left and right sides equals

$$\frac{1}{n}\sum_{i=1}^n \left(\frac{\partial}{\partial \theta}f(z_{i}^{**}, \theta) - \frac{\partial}{\partial \theta}f(z_{i}^{*}, \theta)\right).$$

What would it take for its limit to be zero as $n\to \infty$? One simple (but not necessary) criterion is that for each $\theta$, $\frac{\partial}{\partial\theta}f(z,\theta)$ should be of bounded variation, because--as is immediate from the definition--the magnitude of the difference does not exceed $1/n$ times the total variation of $\frac{\partial}{\partial\theta}f(z,\theta)$, whence its limit would be $0$ (QED).

Indeed, this analysis suggests counterexamples to the original statement. All we need do is construct a family of distributions where these derivatives of the density (with respect to the parameter $\theta$) grow unbounded on the domain $z\in(0,\infty)$. One such family, for $\theta\in (0,1)$, is

$$f(z, \theta) = z^{-\theta} - 1, 0 \le z \le 1; \ f(z,\theta) = 0 \text{ otherwise}.$$

$F(z,\theta)$ is continuously differentiable in $z$ for $z \in (0,\infty)$. If it is desired to make $F$ differentiable at $z=0$, too, then a more complicated counterexample would need to be constructed--but the idea should be clear.


Plots of $f(z,\theta)$ for $\theta\in \{2/3, 1/2, 1/3\}$ are overlaid in this graphic.

An immediate problem crops up: $\partial F(z_0, \theta) / \partial \theta$ is not even defined. We cannot hope for the left hand side even to make sense, much less to be equated to the right hand side--which is finite (equal to $\frac{1}{(1-2 \theta )^2}-\frac{1}{(\theta -1)^2}$) when $0 \lt \theta \lt 1/2$.

  • 1
    $\begingroup$ Thanks wuber, this is indeed a good answer to my question. I worked out another way to prove this using Taylor expansion that required continuous differentiablity of "∂f(z,θ)/∂θ". I think bounded variation is weaker and implied by continuous differentiablity. $\endgroup$ – user41838 Apr 18 '15 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.