Let $X_n$ and $Y_n$ be random variables such that $X_n-Y_n\overset{p}{\longrightarrow}0$ as $n\rightarrow\infty$. Let $f(.)$ be a differentiable function. Is the following correct?

$$f(X_n) = f(Y_n) + f'(Y_n)(X_n-Y_n) + o_p(X_n-Y_n)$$ as $n\rightarrow\infty.$


1 Answer 1


To gain some intuition about the issues, let's consider a sequence of bivariate random variables $(X_n,Y_n)$ that have simple distributions and a function $f$ that is easy to calculate with. We want to give your supposition a hard time in the sense of contemplating functions $f$ for which the Taylor approximation $f(z+a) \approx f(z) + af^\prime(z)$ is particularly poor. If it can withstand such examination then we will proceed to developing a proof; otherwise, we will have constructed a counterexample.

There are many ways to perform such an investigation. For instance, we could examine functions $f$ that, although differentiable, are not continuously differentiable. But perhaps the simplest is to examine functions whose derivative can become arbitrarily large, and then let the sequence $(X_n,Y_n)$ approach points where the derivative "blows up."

Consider, then, the function

$$f:(0,\infty)\to \mathbb{R};\ f(x) = \frac{1}{x},$$

chosen to be easy to compute with but to have a derivative

$$f^\prime(x) = -\frac{1}{x^2}$$

that diverges as $x\to 0.$

Let $(X_n,Y_n)$ be a sequence of discrete random variables converging towards each other and towards $0;$ specifically, suppose

$$\eqalign{ \Pr\left((X_n,Y_n) = \left(\frac{2}{n}, \frac{1}{n}\right)\right) &= \frac{1}{2}\\ \Pr\left((X_n,Y_n) = \left(\frac{3}{n}, \frac{2}{n}\right)\right) &= \frac{1}{2}. }$$

Equivalently, $nY_n-1$ has a Bernoulli$(1/2)$ distribution and $X_n = Y_n + 1/n.$ Their difference converges to zero in any sense you care to use: In particular, since $X_n-Y_n=1/n\to 0$ everywhere, $X_n-Y_n \overset{p}{\longrightarrow} 0.$

Let's compute the difference $f(X_n) - \left[ f(Y_n) + f^\prime(Y_n)(X_n-Y_n) \right].$ This is a quantity you hope will be growing smaller in some sense: you want to be able to assert it is "$o_p(X_n-Y_n).$"

There are two possible values, each with probability $1/2.$ I will show the computations with a table, one row for each possibility:

$$\begin{array}{cc|cccc} X_n & Y_n & f(X_n) & f(Y_n) & f^\prime(Y_n) & f(X_n) - \left[ f(Y_n) + f^\prime(Y_n)(X_n-Y_n) \right] \\ \hline \frac{2}{n} & \frac{1}{n} & \frac{n}{2} & n & -n^2 & \frac{n}{2} - [n - n^2\frac{1}{n}] = \frac{n}{2} \\ \frac{3}{n} & \frac{2}{n} & \frac{n}{3} & \frac{n}{2} & -\frac{n^2}{4} & \frac{n}{3} - [\frac{n}{2} - \frac{n^2}{4}\frac{1}{n}] = \frac{n}{12} \end{array}$$

Because the values $n/2$ and $n/12$ diverge everywhere, there is no possible definition of "$o_p$" that could make the statement true.

The insight provided by this simple example, and variations on it (in which $X_n$ and $Y_n$ may be independent or $f$ may be defined everywhere, for instance) is that for the hoped-for result to be true, you cannot allow $X_n$ and $Y_n$ to have any appreciable probability of attaining values where the Taylor approximation becomes arbitrarily poor.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.