Directly discarding big term in the proof of error propagation formula of variance from random variable $x$ to $f(x)$?

Question

I read the error propagation formula scanario said that, the connection between the variance of a random variable $x$ and $f(x)$ is $\frac{var(f(x))}{|\partial_xf(x)|_{x=\bar{x}}|^2}=var(x)$. While I'm confused in the prove of it in the process of Taylor series:

The proof of it states that, if we define the expected value of random variable $x$ and variance of it as follows: $$ \begin{aligned} \bar{x} &=\sum_{i=1}^{N} p_{i} x_{i} \\ (\Delta x)^{2} &=\sum_{i=1}^{N} p_{i}\left(x_{i}-\bar{x}\right)^{2} \\ &=\overline{x^{2}}-\bar{x}^{2} \end{aligned} $$ And the same for $f(x)$: $$ \begin{aligned} \overline{f(x)} &=\sum_{i=1}^{N} p_{i} f\left(x_{i}\right) \\ (\Delta f)^{2} &=\sum_{i=1}^{N} p_{i}\left(f\left(x_{i}\right)-\bar{f}\right)^{2} \end{aligned}\tag{1} $$ Then we can Taylor expanse it as: $$ \begin{aligned} f\left(x_{i}\right) &=\left.\sum_{n !}^{\infty} \frac{1}{n !}\left(x_{i}-\bar{x}\right)^{n} f^{(n)}(x)\right|_{x=\bar{x}} \\ &\approx f(\bar{x})+\left(x_{i}-\bar{x}\right) f^{(1)}(\bar{x}) \end{aligned}\tag{2} $$ ignoring the square and higher-order term. Then $\overline{f(x)}$ can be written as $$ \begin{aligned} \overline{f(x)} &\approx f(\bar{x})+\overline{\left(x_{i}-\bar{x}\right)} f^{(1)}(\bar{x}) \\ &=f(\bar{x}) \end{aligned}\tag{3} $$ Then replace eq (3) and (2) into (1)， we can get $\frac{var(f(x))}{|\partial_xf(x)|_{x=\bar{x}}|^2}=var(x)$.

My question is, $x_i-\bar{x}$ is not always a small term!!! Why should we just discard it? Or, are there more formal references of this error propagation formula, because all I search online is some other kind of form.

Answer 1

My question is, $x_i-\bar{x}$ is not always a small term!!! Why should we just discard it? Or, are there more formal references of this error propagation formula, because all I search online is some other kind of form.

Indeed $x_i-\bar{x}$ is not always small but the average of it $\overline{x_i-\bar{x}}$ is zero. You could write is as $$\overline{x_i-\bar{x}} = \overline{x_i} - \overline{\bar{x}} = \bar{x_i} - \bar{x}=0$$

---

Sidenote: when you apply this linear approximation then $x_i-\bar{x}$ should actually be small. In this question about a log transformation, you see how the linear approximation works when the deviation are relatively 'small'. (It is also shown how it does not work and how a better approximation can be made)

In that question the following image was used, to show that a normal distribution (or something that looks like a normal distribution) becomes nearly linear transformed with a log transformation, when the deviations are not large.

You can see this similar for the error propagation. With the linear approximation we approximate the curve of the function by a straight line and we use the slope of the function to compute how the deviations changing with an approximately constant factor

Answer 2

The computation like it is shown above discards terms which have higher order than 1 when the function is expanded. Therefore, the proposed formula only applies to functions f which are ("well enough") representable as a linear function. This is the level of error propagation like you will typically find it.

If you want to do error propagation beyond linear taylor expansion because your function f is highly non-linear (or to check the assumptions made during propagation of error) I suggest to go back to the idea of error propagation itself. The following image shows the limitation of using a linearization (resulting in $+/- \Delta y$) instead of all terms of the taylor expansion (resulting in $+\Delta y_+$ and $- \Delta y_-$):

Biggerj1, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons

Picture source: https://commons.wikimedia.org/wiki/File:Propagation_of_error.svg