Do all moments of a random variable need to be well controlled for a valid 2nd order Taylor approximation, or is the third moment sufficient?

Question

In this post, the accepted answer states that we need certain conditions before a second order Taylor series approximation is robust, due to the fact that the variance does not control higher moments.

The author goes on to say that

Therefore, in general, the Taylor series method fails even for 3rd degree polynomials. Iterating this argument shows you cannot expect the Taylor series method to provide accurate results, even for polynomials, unless all moments of your random variable are well controlled.

I'm not fully convinced of this statement and I'm wondering do we really need all moments to be well-controlled for a valid second order Taylor expansion. Because if we look at the formula for the remainder of a Taylor series truncated at second order, such here or here, it only uses the third moment $$ R_2(X) = \frac{f^{(3)}(\zeta)}{3!}E[(X-\mu)^3], $$ and the derivation of this formula does not use the higher moments of the random variable.

So have I overlooked something or is it indeed sufficient to have control of the third moment to ensure the Taylor series approximation is valid? Why would we need control of all moments as the accepted answer in the other post states when the remainder is derived without using higher order moments?

Edit: The formula for the 2nd order Taylor expansion is $$ E[f(X)] = f(\mu) + \frac{1}{2}f''(\mu)E[(X-\mu)^2] + R_2(X). $$

As far as I can see the higher order moments ($E[(X-\mu)^k]$ with $k \ge 4$) have not been used to derive the remainder in that formula so I believe that the formula is accurate as long as only the third moment is well-controlled, e.g. $E[(X-\mu)^3] \ll E[(X-\mu)^2]$.

So am I incorrect or is the linked answer, where the author says all moments of the random variable need to be well controlled, incorrect? If I am incorrect please demonstrate why the higher order moments need to be well-controlled for the above formula to be accurate.

Answer 1

The short answer is that in general, yes, we do need to control all moments (and ensure the function $f$ is sufficiently "nice"). The short reason that the logic above fails is that the location $\zeta$ is a function of $x$.

Take a close look at the definition of the explicit remainder terms in Taylor's theorem. It guarantees only that $\zeta$ lies between $\mu$ and $x$, and so the formula for the expectation of the remainder should be

$$ \mathbb{E}[R_2(X)] = \mathbb{E}\left[ \frac{f^{(3)}(\zeta(X))}{3!}(X-\mu)^3\right]. $$

The term inside the expectation remains a third-degree polynomial in $X$ if and only if $f$ itself is a third-degree polynomial. (Indeed, the remainder term is, by definition, the difference between the function and it's Taylor approximation.)

A simple counterexample

Consider the random variable $X$ with pdf

$$ p_X(x) = c \cdot \frac{1}{1+ x^6} $$

where $c$ is the normalization constant. It's an exercise to check that $X$ has finite first through fourth moments. Now, taking $f(x) = x^3$, we find

$$ \mathrm{Var\left[f\bigl(\frac{1}{n}X\bigr)\right]}=\mathbb{E}\left[f\bigl(\frac{1}{n}X\bigr)^2\right] = \frac{c}{n^3} \int_{-\infty}^\infty \frac{x^6}{1+x^6}\mathrm{d}x = + \infty $$

for any $n\in \mathbb{N}$, while the second-order Taylor approximation is uniformly zero.