Let $A$ and $B$ be random variables and $f(A,B)=\frac{A}{B}$. How should I approximate $E(f(A,B))$? I think a Taylor expansion may be in order, but I am not sure how to fire it off in this function.

My question comes from a practical problem in survey statistics. It may be discussed in textbooks, but I would not know where. Let a sample of size $n$ be taken from an (infinite) population. Not every sample unit may reply to the survey. Let $S$ indicate response ($S=1$) or non-response ($S=0$). The mean estimator $\hat{\mu}=\frac{1}{\sum{S_i}}\sum{S_iy_i}$ thus may be biased due to non-response from the population mean $\mu$. We may be inclined to specify this bias as $$B(\hat{\mu})=E(\frac{1}{\sum{S_i}}\sum{S_iy_i})-\mu$$ for realizations $y_i$ on random variable $Y$. Let $n_r$ be the realized sample size (response sample). I believe $$E(\frac{1}{\sum{S_i}})=\frac{1}{n_r}$$ is only true approximately and relates to the more general problem introduced above, where $A=1$. My specific question is why this equation holds true (approximately).


1 Answer 1


$1/\sum{S_i}$ is a convex function in $\sum{S_i}$. Then by Jensen's inequality

$$E\left(\frac{1}{\sum{S_i}}\right)>\left(\frac{1}{E[\sum{S_i}]}\right) =\frac{1}{n\cdot P(S_i=1)}$$

the last equality if we assume that each respondent has an equal probability to respond or not. An estimator of this probability is the sample proportion $\hat P(S_i=1) =n_r/n$ so we get

$$est.\left(\frac{1}{E[\sum{S_i}]}\right) =\frac 1{n_r}$$

So "between" $E\left(\frac{1}{\sum{S_i}}\right)$ and $1/n_r$ there exists both the distance due to the non-linearity, as well as the estimation error. The estimation error can go either way, so we cannot conclude on the final relation between the two.

Nevertheless appealing to asymptotics, $\hat P(S_i=1) \xrightarrow{p} P(S_i=1)$ and $E\left(\frac{1}{\sum{S_i}}\right)\rightarrow \left(\frac{1}{E[\sum{S_i}]}\right)$, so for "large samples" we accept $1/n_r$ as an approximation to $E\left(\frac{1}{\sum{S_i}}\right)$.

But for the general case, the situation changes. Write $w_i \equiv S_i/\sum S_i$ and so $\sum w_i =1$, and we have

$$\hat \mu = \sum w_iY_i$$

If we assume that
a) $S_i$ and $Y_i$ are independent,(i.e. that whether somebody responds or not does not depend on his own value of $Y$ -and this is not always the case, for example think of a survey that asks something "sensitive", say "what is your monthly income"? People with high income may choose not to respond rather than record a true or false statement), and taking into account that

b) all members of the population are identically distributed as random variables, and have the common mean $E(Y_i) =\mu, \; \forall i$, then

$$E(\hat \mu) = \sum E(w_i)E(Y_i) = \mu\cdot\sum E(w_i) = \mu\cdot E\left(\sum w_i\right) = \mu\cdot E(1) = \mu$$

so the estimator is, after all, unbiased. This depends crucially on the independence assumption between $S_i$ and $Y_i$ because it implies that the sub-sample of those that responded remains a random sample from, a "representative" sample of, the population, and so its sample average is still an unbiased estimator.


Regarding the Taylor series expansion, for the function $1/Z$ it is, around some center $z_0$,

$$E(1/Z) = \frac 1{z_0} - \frac 1{z_0^2}[E(Z) - z_0] + \frac 1{z_0^3}E[Z - z_0]^2 + E(R_2)$$

$\sum S_i$ is a binomial random variable. So centering on $z_0=E(\sum S_i) = np_s$ we have

$$E\left(\frac{1}{\sum{S_i}}\right) = \frac 1{np_s}-\frac 1{n^2p_s^2}(E(S_i)-np_s)+\frac 1{n^3p_s^3}\text{Var}(S_i) +E(R_2)$$

$$=\frac 1{np_s} -0+\frac {np_s(1-p_s)}{n^3p_s^3} + E(R_2) = \frac 1{np_s} + O(n^{-2})$$

Since $\hat p_s = n_r/n$ we arrive at

$$\hat E\left(\frac{1}{\sum{S_i}}\right) \approx \frac 1{n_r}$$

  • $\begingroup$ Thank you - I once heard that Taylor expension plays a role, too. In particular I think we may be able to write $E(\frac{1}{\sum{S_i}})$ as a Taylor series. I believe it should be shown that $\frac{1}{n_r}$ is the major term, whereas the remainder is negligible (maybe in particular in large samples?). Are you aware of anything like this? $\endgroup$
    – tomka
    Commented Oct 22, 2014 at 11:53
  • 1
    $\begingroup$ Added the case. $\endgroup$ Commented Oct 22, 2014 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.