9
$\begingroup$

I am reading a text, "Mathematical Statistics and Data Analysis" by John Rice. We are concerned with approximating the expected value and variance of the random variable $Y$. We are able to calculate the expected value and variance of the random variable $X$ and we know the relation $Y = g(X)$. So, it's possible to approximate the expected value and variance of $Y$ using the Taylor series expansion of $g$ about $\mu_X$.

On page 162, he lists 3 equations.

  1. The expected value of $Y$ using the 1st-Order Taylor series expansion. It is: $\mu_Y \approx g(\mu_X)$. This is referred to later in my question as $E(Y_1)$.

  2. The variance of $Y$ using the 1st-Order Taylor series expansion. It is: $\sigma_Y^2 \approx \sigma_X^2 (g'(\mu_X))^2$. This is referred to later in my question as $Var(Y_1)$.

  3. The expected value of $Y$ using the 2nd-Order Taylor series expansion. It is $\mu_Y \approx g(\mu_X) + \frac12 \sigma_X^2 g''(\mu_X)$. This is referred to later in my question as $E(Y_2)$.

Note that there are two different expressions for $Y$ because we are using two different orders in the Taylor series expansion. Equations 1 and 2 refer to $Y_1 = g(X) \approx g(\mu_X) + (X-\mu_X)g'(\mu_X)$. Equation 3 refers to $Y_2 = g(X) \approx g(\mu_X) + (X-\mu_X)g'(\mu_X) + \frac12 (X-\mu_X)^2 g''(\mu_X)$.

Note that specifically the equation for $Var(Y_2)$ is not given. Later, the author seems to use the equation for the variance of $Y_1$ (Equation 2), when in fact he is referring to expected value of $Y_2$ (Equation 3). This seems to imply $Var(Y_2) = Var(Y_1)$.

I have tried to calculate by hand $Var(Y_2)$, and I am getting a somewhat complicated expression. Here is my work (I stopped because at the end I am getting $X^3$ terms in the expectation): $$ \begin{aligned} Var(Y_2) &= E[( g(\mu_X) + (X-\mu_X)a + \frac12 (X-\mu_X)^2 b - g(\mu_X) - \frac12 \sigma_X^2 b )^2] \\ &= E\left[((X-\mu_X)a + \left(\frac12 (X-\mu_X)^2 - \frac12 \sigma_X^2)b\right)^2\right] \\ &= E\left[(ca + \left(\frac12 c^2 - \frac12 \sigma_X^2)b\right)^2\right] \\ & = E[c^2 a^2 + ca(c^2 - \sigma_X^2)b + \frac14(c^2-\sigma_X^2)^2 b^2] \\ & = E[(X^2 - 2X \mu_X + \mu_X^2)a^2 + (X-\mu_X)a((X^2 - 2X\mu_X + \mu_X^2) - \sigma_X^2)b \\ & + \frac14((X^2 - 2X\mu_X + \mu_X^2)-\sigma_X^2)^2 b^2] \end{aligned} $$

Note that in the above equations, $a = g'(\mu_X)$, $b = g''(\mu_X)$, and $c = X-\mu_X$. What is $Var(Y_2)$?

Thanks.

$\endgroup$
  • $\begingroup$ Why did you stop at $X^3$? Because the second-order approximation is a quadratic function of $X$, its variance will generally involve moments of $X$ up to $2 \cdot 2 = 4$. The third moment may be zero, but the fourth moment is definitely going to show up and not get canceled out by anything. $\endgroup$ – whuber Jul 13 '11 at 21:11
8
$\begingroup$

Assuming $Y=g(X)$, we can derive the approximate variance of $Y$ using the second-order Taylor expansion of $g(X)$ about $\mu_X=\mathbf{E}[X]$ as follows:

$$\begin{eqnarray*} \mathbf{Var}[Y] &=& \mathbf{Var}[g(X)]\\ &\approx& \mathbf{Var}[g(\mu_X)+g'(\mu_X)(X-\mu_X)+\frac{1}{2}g''(\mu_X)(X-\mu_X)^2]\\ &=& (g'(\mu_X))^2\sigma_{X}^{2}+\frac{1}{4}(g''(\mu_X))^2\mathbf{Var}[(X-\mu_X)^2]\\ & & +g'(\mu_X)g''(\mu_X)\mathbf{Cov}[X-\mu_X,(X-\mu_X)^2]\\ &=& (g'(\mu_X))^2\sigma_{X}^{2}+\frac{1}{4}(g''(\mu_X))^2\mathbf{E}[(X-\mu_X)^4-\sigma_{X}^{4}]\\ & & +g'(\mu_X)g''(\mu_X)\left(\mathbf{E}(X^3)-3\mu_X(\sigma_{X}^{2}+\mu_{X}^{2})+2\mu_{X}^{3}\right)\\ &=& (g'(\mu_X))^2\sigma_{X}^{2}\\ & & +\frac{1}{4}(g''(\mu_X))^2\left(\mathbf{E}[X^4]-4\mu_X\mathbf{E}[X^3]+6\mu_{X}^{2}(\sigma_{X}^{2}+\mu_{X}^{2})-3\mu_{X}^{4}-\sigma_{X}^{4}\right)\\ & & +g'(\mu_X)g''(\mu_X)\left(\mathbf{E}(X^3)-3\mu_X(\sigma_{X}^{2}+\mu_{X}^{2})+2\mu_{X}^{3}\right)\\ \end{eqnarray*}$$

As @whuber pointed out in the comments, this can be cleaned up a bit by using the third and fourth central moments of $X$. A central moment is defined as $\mu_k=\mathbf{E}[(X-\mu_X)^k]$. Notice that $\sigma_{X}^{2}=\mu_2$. Using this new notation, we have that $$\mathbf{Var}[Y]\approx(g'(\mu_X))^2\sigma_{X}^{2}+g'(\mu_X)g''(\mu_X)\mu_3+\frac{1}{4}(g''(\mu_X))^2(\mu_4-\sigma_{X}^{4})$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ That's the right approach, but didn't you forget to include the covariance between $X-\mu_X$ and $(X-\mu_X)^2$? $\endgroup$ – whuber Jul 13 '11 at 22:39
  • $\begingroup$ @whuber Yes I did. Thanks for pointing that out. I will edit this soon. $\endgroup$ – assumednormal Jul 13 '11 at 22:41
  • $\begingroup$ You can save yourself some trouble by writing the answer in terms of the second, third, and fourth central moments, $\sigma^2$, $\mu_3$, and $\mu_4$. You should obtain $\sigma^2 g'(\mu )^2 + \mu_3 g'(\mu ) g''(\mu )+\frac{1}{4} \left(\mu_4-\sigma ^4\right) g''(\mu )^2$. $\endgroup$ – whuber Jul 13 '11 at 23:01
  • $\begingroup$ @jrand - My apologies. I didn't realize you had this in your original post. I'm not deleting my post, though, because it took a while to typeset. $\endgroup$ – assumednormal Jul 15 '11 at 3:35
  • $\begingroup$ @Max, whuber: Thank you for the answer and explanation. $\endgroup$ – jrand Jul 16 '11 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.