4
$\begingroup$

In the one-dimensional case, if $X$ is $\mathcal{N}(\mu,\sigma^2)$, then $Y =\alpha X + \beta $ is $\mathcal{N}(\alpha \mu + \beta,\alpha^2\sigma^2)$ . We can prove this using the cumulative distribution function of of $Y$

$F_Y(a) = P\{Y \leq a\} = P\{\alpha X + \beta \leq a\} = P\{X \leq (a-\beta)/\alpha\}$.

Substituting $Y =\alpha X + \beta $ and change of variable gives us,

$F_Y(a) = \int_{-\infty}^{a} \frac{1}{\sqrt{2\pi}(\alpha\sigma)} \exp \{ \frac{-(v-(\alpha \mu + \beta))^2}{2(\alpha\sigma)^2}\} dv $

Hence $f_Y(v) = \frac{1}{\sqrt{2\pi}(\alpha\sigma)} \exp \{ \frac{-(v-(\alpha \mu + \beta))^2}{2(\alpha\sigma)^2}\} $

Thus $Y$ is $\mathcal{N}(\alpha \mu + \beta, \alpha^2 \sigma^2)$.

In the multivariate case, if $X$ is $\mathcal{N}(\mu,\Sigma)$ and $Y=\alpha X + \beta$, is $Y \sim \mathcal{N}(\alpha \mu + \beta,\alpha^2\Sigma)$? If so, how do we prove it?

$\endgroup$

1 Answer 1

8
$\begingroup$

The method of characteristic functions (CF) will work here. So we have the CF for $X$ as $$\varphi_{X}(t)=\exp\left(it^{T}\mu_{X}-\frac{1}{2}t^{T}\Sigma_{X}t\right)$$ Now we make the substitution $Y=\alpha X + \beta$ in the CF and we get:

$$\varphi_{Y}(t)=E\left[\exp(it^{T}Y)\right]=E\left[\exp(it^{T}\alpha X +it^{T}\beta)\right]=\exp(it^{T}\beta)\varphi_{X}(\alpha t)$$

Then substitute in the CF expression for $X$.

$$\varphi_{Y}(t)=\exp(it^{T}\beta)\exp\left(i(\alpha t)^{T}\mu_{X}-\frac{1}{2}(\alpha t)^{T}\Sigma_{X}(\alpha t)\right)$$ $$=\exp\left(it^{T}[\alpha\mu_{X}+\beta]-\frac{1}{2}t^{T}[\alpha^{2}\Sigma_{X}]t\right)$$

But this is the characteristic function of a new normal distribution with mean vector $\alpha\mu_{X}+\beta$ and covariance matrix $\alpha^{2}\Sigma_{X}$. As characteristic functions are uniquely defined from a distribution function and vice versa, you have your proof.

To generalise to the case where $\alpha$ is an appropriately defined $c\times p$ matrix ($p$ is the dimension of $X$). we simply replace the covariance matrix $\alpha^{2}\Sigma_{X}$ with the $c\times c$ covariance matrix $\alpha\Sigma_{X}\alpha^{T}$. Note that $\beta$ must be a $c\times 1$ vector for mean vector to make sense - but it is unchanged at $\alpha\mu_{X}+\beta$.

$\endgroup$
17
  • 1
    $\begingroup$ @Arun, it should be noted that this result holds much more generally, in particular, when $\alpha$ is a matrix (of appropriate dimensions). Only the expression for the covariance chances slightly to accommodate this. Note that @probabilityislogic is (rightly) treating $\beta$ as an arbitrary vector of the correct dimension here. $\endgroup$
    – cardinal
    Apr 25, 2011 at 11:43
  • 1
    $\begingroup$ (+1) A couple minor quibbles: (a) Generalize to the matrix case mentioned above, (b) The distribution and not the pdf is characterized by the ch.f., so the last sentence should be changed and (c) Using \exp will get $\exp$ to render properly. $\endgroup$
    – cardinal
    Apr 25, 2011 at 13:22
  • 1
    $\begingroup$ @probabilityislogic: Certainly the characteristic function is uniquely defined once the density is specified. The reverse is not (quite) true. First there is the issue of the existence of the pdf in the first place. Second, we can change the pdf on any set of measure zero and obtain the same characteristic function. Note that this includes dense(!) subsets of $\mathbb{R}$, for example, the rationals. The distribution function is uniquely determined by the ch.f., though. $\endgroup$
    – cardinal
    Apr 26, 2011 at 0:15
  • 1
    $\begingroup$ @probabilityislogic, the pdf need not exist for the ch.f. to exist. Indeed, a ch.f. exists for all distribution functions $F$ (not just those with densities). This is a simple consequence of monotonicity of expectation, i.e., $|\mathbb{E} e^{i X}| \leq \mathbb{E}|e^{iX}| \leq 1$. $\endgroup$
    – cardinal
    Apr 26, 2011 at 2:38
  • 1
    $\begingroup$ @probabilityislogic, For an example of changing a pdf on a dense subset of the real line, let $\varphi(x)$ be the density of a standard normal and define $\bar{\varphi}(x) = \varphi(x)$ for all $x \not\in \mathbb{Q}$ and $\bar{\varphi}(x) = \varphi(x) + 1/q(x)$ for every $x \in \mathbb{Q}$ where $q(x)$ is the denominator of $x$ expressed in lowest terms. Then $\bar{\varphi}(x)$ is continuous at every irrational number $x$ and discontinuous at every rational number, yet the characteristic functions of $\varphi$ and $\bar{\varphi}$ coincide. $\endgroup$
    – cardinal
    Apr 26, 2011 at 2:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.