I have the following hierarchical model:

$y_{i} = \alpha + \beta_{i}x_{i} + \varepsilon_{i} $ where $\varepsilon \sim N(0,\sigma^{2})$.

$\beta_{i} \sim N(\gamma x_{i},\sigma^{2}x_{i}^{-2})$

Knowing this information I would like to understand why we can redifine the prior of $\beta_{i}$ as following:

$\beta_{i} = \gamma x_{i} + \sqrt{\sigma^{2}x_{i}^{-2}}\eta_{i}$ with $\eta_{i} \sim N(0,1)$.

I don't understand the intuition and the mathematical property/derivation that allows us to express it as the mean plus square root of the variance. Some clarification on this would be really helpful!

  • $\begingroup$ Can you provide reference for those formulas? $\endgroup$ – Tim Jun 19 '17 at 12:20
  • $\begingroup$ @Tim it's extracted from a professor's exercise someone gave me, no reference possibly sorry. $\endgroup$ – adrian1121 Jun 19 '17 at 12:23
  • $\begingroup$ i dont know if i get you correct, but it is just a scaleing of the standard normal distribution. i.e., if $\eta_i \sim N(0,1)$ then $\beta_i = \mu_i + \sigma_i \eta_i \sim N(\mu_i, \sigma_i^2)$. here you have $\mu_i= \gamma x_i$ and $\sigma_i = \sqrt{\sigma x_i^{-2}}$. see for example en.wikipedia.org/wiki/… $\endgroup$ – chRrr Jun 19 '17 at 12:39

Your last line could also have been written like this: $$ \beta_i = \gamma x_i+\zeta_i $$ with: $$ \zeta_i\sim N\left(0,\sigma^2x_i^{-2}\right) $$ Any variable with a scaled Normal distribution can be rewritten as a product between a variable with a standard (i.e. unit variance) Normal distribution, and some scaling factor equal to the standard deviation of the desired scaled Normal. In your case $\sqrt{\sigma^2x_i^{-2}}$ is the standard deviation of the scaled Normal that describes the variability in $\beta_i$, such that: $$ \zeta_i=\eta_i\sqrt{\sigma^2x_i^{-2}} $$ If we substitute this into the equation at the top of my answer, you get back the equation in your question.


Regarding the lines $$\beta_i\sim\dots$$ to $$\beta_i=\dots$$ it is just about the normalization of the normal distribution (note that variance is square of standard deviation).


As I see it, your question is why a random variable $X \sim N(\mu, \sigma^2)$ is equivalent to $Y = \mu + \sigma Z$ with $Z \sim N(0,1)$.

A very informal explanation is as follows. Consider a standard normal random variable $Z \sim N(0,1)$ and then let $Y = \mu + \sigma Z$ (with $\sigma > 0$). If you think about the Bell curve, you'll see that the transformation of multiplying by $\sigma$ and adding $\mu$, then this keeps the Bell curve intact, it just moves it so that its center is at $\mu$ and expands it such that its width is inflated (or shrunk, for $\sigma < 1$) by $\sigma$. Now all we need to do is make sure that the mean and variance of $Y$ are as expected. As you see from the Wikipedia links below, these are standard properties of the mean and variance.

https://en.wikipedia.org/wiki/Expected_value#Properties https://en.wikipedia.org/wiki/Variance#Properties

We have

$E(Y) = E(\mu + \sigma Z) = \mu + \sigma E(Z) = \mu + 0 = \mu$

$Var(Y) = Var(\mu + \sigma Z) = \sigma^2 Var(Z) = \sigma^2$

so $Y \sim N(\mu, \sigma^2)$, as required.

edit: added the explanation below

A slightly more formal way to show this is through the distribution function. Let $X, Y, Z$ be as above, and let $a$ be a real number. Then

$$ P(Y \leq a) = P(\mu + \sigma Z \leq a) = P(Z \leq \frac{a - \mu}{\sigma}) = $$

$$ = \int_{-\infty}^{\frac{a - \mu}{\sigma}} \frac{1}{\sqrt{2\pi}} \mathrm{exp}(-\frac{z^2}{2})dz $$

Similarly, we have

$$ P(X \leq a) = \int_{-\infty}^a \frac{1}{\sigma \sqrt{2\pi}} \mathrm{exp}(-\frac{1}{2} (\frac{x - \mu}{\sigma})^2)dx $$

In this final integral, simply apply the substitution $z = \frac{x - \mu}{\sigma}$ to see that they are the same. In other words, $P(Y \leq a) = P(X \leq a)$ for all real numbers $a$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.