4
$\begingroup$

Let's say I have a normal variable $X$. I feed $X$ into some function, which applies varies mathematical operations to it - for instance, addition, subtraction, multiplication, division, $x^r$, $e^x$, trigonometric functions, etc.

What happens to the standard deviation?

Addition and subtraction seems trivial: If you imagine the data plotted as a scatter plot, $X+nq$ is just $X$ with the vertical axis moved by $q$. Since standard deviation characterizes the spread of the point, and moving the axis does not change the shape of the plot, these two operations will not change standard deviation.

By the same logic, multiplication clearly scales the standard deviation by the same amount as the axis. One could literally draw the distribution on paper, mark the standard deviation as a line segment, then erase the axis labels and write the scaled ones and obtain the adjusted standard deviation.

However, for powers this doesn't work: For example, with $e^x$ the points that are farther away from the axis become more spread out, and the ones below become less spread out. In fact, $e^x$ will change the mean and skew of the distribution, and the histogram will look completely different - thus it is not surprising that $std(f(x))$ is not just $f(std(x))$.

For trigonometric functions, my intuition breaks down completely. I don't even know how to interpret it in a useful away. I know I can simply write out the equation describing the normal distribution, apply the trigonometric function, do the algebra and go from there, but intuitively I am stuck.

Are there broad, easily determinable classes of functions that "leave the standard deviation alone" and functions that warp it in strange ways? Is there an easy way to tell, without doing a lot of algebra, if a given function's standard deviation can be easily related to the standard deviation of its argument?

Motivation: In physical sciences, one often ends up some measured quantity being mathematically related to some other measured quantities. For example, the period of a swinging pendulum is approximated by:

$$T \approx 2\pi\sqrt{\frac{L}{g}}$$

This can even be derived by simple unit analysis. But if one wants to experimentally confirm (or reject) this rule, there is a practical challenge besides constructing the experimental apparatus: None of the inputs ($L$ and $g$, though $\pi$ is also problematic, albeit more exotically so) can be known precisely. They must be measured, and the measurement will have some error (often thought to have normal-like character).

The question is, then, how does error in $L$ or $g$ affect the error of the predicted value for $T$ (let's ignore the other problem of actually measuring $T$)? More importantly, can one easily decide this without extensive calculation? Consider, for instance, an experimentalist who finds himself thinking: "Do I really have to go all the way upstairs to fetch the ruler? Will it ruin the whole calculation if I just ballpark the rope length?"

Note how solving the problem analytically is not an option here: You could easily go fetch the ruler in a fraction of the time and be done with it, if that was the best option.

Another example: The Drake equation uses simple multiplication to estimate an unfamiliar variable from well-known quantities (the so called Fermi problem). Supposedly, because the equation is a product, even if there is some variance in the estimates for the parameters, the result of the equation can be very accurate. Again the critical question is (I think), how does the variance in inputs affect the variance of the outputs?

Stated in yet another way, my question is: When applying a mathematical model to empirical data, can you easily tell which measurements really have to be very precise, but which ones are okay if they are off by a bit?

$\endgroup$
6
$\begingroup$

What happens to the standard deviation?

The most general way to do this is to transform the variable and compute the standard deviation of the transformed variable.

For example, if $X\sim N(\mu,\sigma^2)$, then $e^X\sim\text{logN}(\mu,\sigma^2)$, which has s.d. $e^{\mu+\sigma^2/2}.\sqrt{e^{\sigma^2}-1}$.

However, rather than having to do the transformation, we can also use the law of the unconscious statistician to compute the variance. Or there are a variety of other approaches that can sometimes be used.

Another possibility is to use Taylor series to attempt to approximate the variance of the transformed variable

... and I expect that's what you're actually after.

With that, to first order, the variance of the transformed variable can approximately be written in terms of the variance of the original variable:

$\operatorname{var}\left[f(X)\right]\approx \left(f'(\operatorname{E}\left[X\right])\right)^2\operatorname{var}\left[X\right] = \left(f'(\mu_X)\right)^2\sigma^2_X\,,$

so

$\operatorname{sd}\left[f(X)\right]\approx f'(\mu_X)\,\sigma_X\,.$

This expression can be used to get some intuitive sense of how the standard deviation changes as we transform the variable.

However, you must exercise some caution -- it's easy to get oneself into a deal of trouble just assuming this will always work.

See also this, especially this part.

$\endgroup$
  • $\begingroup$ Thank you for the great answer! Both LOTUS and the other method was exactly what I was looking for. $\endgroup$ – Superbest Nov 19 '14 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.