If $X \sim \text{N}(0, \sigma^2)$ it is well-known that we have the second-order Taylor approximation:

$$\mathbb{V}[f(X)] \approx f'(\mu)^2 \cdot \sigma^2 + \frac{f''(\mu)^2}{2} \cdot \sigma^4.$$

What is the general form for this variance approximation when we have a known underlying distribution, but we do not assume normality or zero mean?

  • $\begingroup$ What is the function $f$ here? $\endgroup$
    – Nick
    Mar 4 '20 at 3:23
  • 1
    $\begingroup$ Generally $f$ would simply be any twice differentiable function. The goal is to provide a general approximating form that can be used on any function of this kind. $\endgroup$
    – Ben
    Mar 4 '20 at 3:44

The general form of the variance approximation depends on the first four moments of the distribution. To facilitate our analysis, we suppose that $X$ has mean $\mu$, variance $\sigma^2$, skewness $\gamma$ and kurtosis $\kappa$, and assume that the kurtosis is finite. The general variance approximation is:

$$\boxed{\begin{equation} \begin{aligned} \mathbb{V}[f(X)] &\approx ( f''(\mu)^2 \mu^2 - f'(\mu)f''(\mu) \mu + f'(\mu)^2 ) \cdot \sigma^2 \\[6pt] &\quad - \frac{f''(\mu)(f'(\mu) + \mu f''(\mu))}{2} \cdot \gamma \sigma^3 + \frac{f''(\mu)^2}{4} \cdot (\kappa-1) \sigma^4. \\[6pt] \end{aligned} \end{equation}}$$

The special case shown in the question is for a centred normal distribution, where we have $\mu=0$, $\gamma=0$ and $\kappa=3$.

Working for this result: Let's start by taking the second-order Taylor expansion of $f$ around the mean $\mu$. This gives us the approximating quadratic function:

$$\begin{equation} \begin{aligned} \hat{f}(x) &= f(\mu) + f'(\mu) \cdot (x-\mu) + \frac{1}{2} \cdot f''(\mu) \cdot (x-\mu)^2 \\[6pt] &= \frac{f''(\mu)}{2} \cdot x^2 + (f'(\mu) - \mu f''(\mu)) \cdot x + \text{const}. \\[6pt] \end{aligned} \end{equation}$$

The variance of this quadratic function is:

$$\begin{equation} \begin{aligned} \mathbb{V}[\hat{f}(X)] &= \mathbb{V} \Big[ \frac{f''(\mu)}{2} \cdot X^2 + (f'(\mu) - a f''(\mu)) \cdot X \Big] \\[6pt] &= \frac{f''(\mu)^2}{4} \cdot \mathbb{V}[X^2] + (f'(\mu) - \mu f''(\mu))^2 \cdot \mathbb{V}[X] \\[6pt] &\quad + \frac{f''(\mu)}{2} \cdot (f'(\mu) - \mu f''(\mu)) \cdot \mathbb{C}[X,X^2]. \\[6pt] \end{aligned} \end{equation}$$

Using the relationships between the moments and cumulants, it can be shown (see here and here for derivation) that:

$$\begin{equation} \begin{aligned} \mathbb{V}[X] &= \sigma^2, \\[6pt] \mathbb{V}[X^2] &= 4 \mu^2 \sigma^2 - 4 \mu \gamma \sigma^3 + (\kappa-1) \sigma^4, \\[6pt] \mathbb{C}[X,X^2] &= 2 \mu \sigma^2 + \gamma \sigma^3. \\[6pt] \end{aligned} \end{equation}$$

We therefore have:

$$\begin{equation} \begin{aligned} \mathbb{V}[\hat{f}(X)] &= \frac{f''(\mu)^2}{4} \cdot \Big[ 4 \mu^2 \sigma^2 - 4 \mu \gamma \sigma^3 + (\kappa-1) \sigma^4 \Big] + (f'(\mu) - \mu f''(\mu))^2 \cdot \sigma^2 \\[6pt] &\quad + \frac{f''(\mu)}{2} \cdot (f'(\mu) - \mu f''(\mu)) \cdot \Big[ 2 \mu \sigma^2 + \gamma \sigma^3 \Big] \\[6pt] &= f''(\mu)^2 \mu^2 \sigma^2 + (f'(\mu) - \mu f''(\mu))^2 \sigma^2 + (f'(\mu) - \mu f''(\mu)) f''(\mu) \mu \sigma^2 \\[6pt] &\quad - f''(\mu)^2 \mu \gamma \sigma^3 + (f'(\mu) - \mu f''(\mu)) \frac{f''(\mu)}{2} \gamma \sigma^3 \\[6pt] &\quad + \frac{f''(\mu)^2}{4} (\kappa-1) \sigma^4 \\[6pt] &= \Big[ f''(\mu)^2 \mu^2 + (f'(\mu) - \mu f''(\mu)) f''(\mu) \mu + (f'(\mu) - \mu f''(\mu))^2 \Big] \sigma^2 \\[6pt] &\quad - \Big[ f''(\mu)^2 \mu + (f'(\mu) - \mu f''(\mu)) \frac{f''(\mu)}{2} \Big] \gamma \sigma^3 \\[6pt] &\quad + \frac{f''(\mu)^2}{4} (\kappa-1) \sigma^4 \\[6pt] &= \Big[ f'(\mu) f''(\mu) \mu + (f'(\mu) - \mu f''(\mu))^2 \Big] \sigma^2 \\[6pt] &\quad - \Big[ f''(\mu)^2 \mu + (f'(\mu) - \mu f''(\mu)) \frac{f''(\mu)}{2} \Big] \gamma \sigma^3 \\[6pt] &\quad + \frac{f''(\mu)^2}{4} (\kappa-1) \sigma^4. \\[6pt] \end{aligned} \end{equation}$$

This gives us the true variance of the second-order Taylor approximation to the function. Observe that the approximation depends on the skewness and kurtosis of the distribution, not just its mean and variance. Now, using this variance as the approximation to the variance of the actual function (i.e., taking $\mathbb{V}[f(X)] \approx \mathbb{V}[\hat{f}(X)]$) gives the result shown above.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.