This answer notes that if a programming language/libraries provide a procedure that returns random samples from a standard normal distribution, we can generate samples from another normal distribution with the same mean by multiplying the samples by the standard deviation $\sigma$ of the desired distribution.

This seems to work. For example, in R, these histograms produced by these two lines of code using the rnorm function, which generates samples from a normal distribution, are visually indistinguishable:

hist(rnorm(100000, sd=0.5), xlim=c(-3,3), breaks=50)
hist(0.5*rnorm(100000),     xlim=c(-3,3), breaks=50)

I don't understand why it works.

In both the normal probability density function and the cumulative distribution function, $\sigma$ appears, squared, in the argument of an exponential function.

Why should simply multiplying by standard deviation turns samples of the standard normal into samples of a distribution with that standard deviation? (It's not surprising that multiplying the standard normal PDF by a constant doesn't produce a PDF of the normal distribution with that standard deviation.)

(If the answer is closely related: For what classes of probability distributions does multiplying samples by a constant generate samples with a distribution whose standard deviation is that multiple of the original distribution's sd?)

  • 4
    $\begingroup$ When you multiply all values by a constant, you're just changing your units of measurement. For instance, if the original distribution describes distances in miles, then multiplying all values by 1.609 describes the same distances in kilometers. It's exactly the same distribution. You should now be able to answer your last question using analogous reasoning. Mathematically, you should be noticing that the argument of the exponential in the PDF is a function of $x/\sigma$, not just $x$ or $\sigma$ alone, and that the differential element is actually $d(x/\sigma)=dx/\sigma$. $\endgroup$ – whuber Dec 19 '15 at 4:59

Assume that $X$ has a normal distribution with mean $\mu=0$ and variance $\sigma^2$. Then the probability density function (pdf) of the random variable $X$ is given by:

\begin{eqnarray*} f_X(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{x^{2}}{2\sigma^{2}}} \end{eqnarray*}

for $-\infty<x<\infty$ and $\sigma>0$. Now, when $Z$ has a standard normal distribution, $\mu=0$ and $\sigma^2=1$, so, it's pdf is given by:

\begin{eqnarray*} f_Z(z)=\frac{1}{\sqrt{2\pi}}e^{-\frac{x^{2}}{2}} \end{eqnarray*}

for $-\infty<z<\infty$. If we then multiply $Z$ by the standard deviation $\sigma$ and let that be equal to the function $g(Z)$, (i.e. $Y=g(Z)=\sigma Z$) we can use the formula for transforming functions of random variables (see Casella and Berger (2002), Theorem 2.1.8): \begin{eqnarray*} {f_Y(y)=f_Z(z)(g^{-1}(y))}{{d\over{dy}}{g^{-1}(y)}} \end{eqnarray*} First we find $Z=g^{-1}(y)={y\over{\sigma}}$ and ${d\over{dy}}{g^{-1}(y)}={1\over{\sigma}}$.

So, substituting these terms, we have:

\begin{eqnarray*} f_{Y}(y) & = & f_{Z}\left(g^{-1}(y)\right){\frac{d}{{dy}}{g^{-1}(y)}}\\ & = & f_{Z}\left(\frac{1}{\sigma}\right)\frac{1}{\sigma}\\ & = & \frac{1}{{\sqrt{2\pi}}}{e^{-}\frac{\left(\frac{1}{\sigma}\right)^{2}}{{2}}}\left(\frac{1}{\sigma}\right)\\ & = & \frac{1}{{\sqrt{2\pi}}}\left(\frac{1}{\sigma}\right){e^{-}\frac{1}{{2\sigma^{2}}}}\\ & = & \frac{1}{{\sqrt{2\pi}\sigma}}{e^{-\frac{1}{{2\sigma^{2}}}}} \end{eqnarray*}

This is simply the PDF of $f_X$ given at the beginning of the proof which is simply the pdf of a normal random variable with mean $\mu=0$ and variance $\sigma^2$. Hence, $Y\sim N(0, \sigma^2)$. So if you look closely back through the proof, you'll see that the squared $\sigma$ exponent term is introduced through the original squared $x$ term via composite functions with the inner function being the inverse of transformation. So this is how multiplying by $\sigma$ introduces $\sigma^2$ into the the pdf.

| cite | improve this answer | |

The normal CDF can be written as $$p=\frac{1}{2}\left[1+\text{erf}\left(\frac{x-\mu}{\sigma\sqrt{2}}\right)\right]$$ where $\text{erf}$ is the error function. For a standard normal, $\mu=0$ and $\sigma=1$. If you were to multiply your random variate $x$ by constant $a$, the only way in which you could keep the cumulative probability $p$ from changing would be to multiply the same constant by the standard deviation. $$p=\frac{1}{2}\left[1+\text{erf}\left(\frac{a(x-\mu)}{a \sigma\sqrt{2}}\right)\right]$$

| cite | improve this answer | |
  • $\begingroup$ That seems to me to be a (partial) restatement of the problem: Multiplying samples by a constant gives samples from a distribution with the sd scaled in the same way, but an analogous operation on the cdf or pdf does not. (Or: What, exactly, is the properly analogous operation?) $\endgroup$ – Mars Dec 19 '15 at 5:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.