Skip to main content
english
Source Link
Yves
  • 5.7k
  • 1
  • 23
  • 38

Assume that $\mu$ and $\Sigma$ are a priori independent and that $y$ has a normal margin with mean $\mu_0$ and variance $\Sigma_0$. I will prove that then the variance $\Sigma$ must be constant, and the mean $\mu$ must have a normal prior (possibly degenerate).

I will stick to the one-dimensional case for simplicity, using the characteristic function (c.f.) of $y$, i.e. $\phi_y(t) := \mathbb{E}[e^{yit}]$. We know that $\phi_y(t) = \exp\{\mu_0 it - \Sigma_0 t^2 /2$} and a similar formula holds for the distribution of $y$ conditional on $\mu$ and $\Sigma$, which is normal by assumption. So for any real $t$ $$ \mathbb{E}[e^{yit}] = \int \mathbb{E}\left[e^{yit} \, \vert \,\mu,\,\Sigma\right]\, p(\mu) p(\Sigma) \,\text{d}\mu \text{d} \Sigma = \int \exp\left\{ \mu it - \Sigma t^2/2 \right\} \,p(\mu) p(\Sigma) \,\text{d}\mu \text{d}\Sigma, $$ and by rearranging the integral, we must have $$ \exp\left\{ \mu_0 it - \Sigma_0 t^2 /2 \right\} = \left[\int \exp\left\{ \mu it \right\} p(\mu) \,\text{d}\mu \right] \left[\int \exp\left\{ -\Sigma t^2/2\right\} p(\Sigma) \,\text{d}\Sigma \right]. $$ The assumptions needed for such a rearrangement are easily checked.

The first integral at right hand side, say $\phi_1(t)$, is the c.f. of $\mu$. Note that since $\phi_1(t) e^{-\mu_0 it}$ is found to be real, we see that the distribution of $\mu$ is symmetric w.r.t. $\mu_0$, and hence that $\mathbb{E}[\mu] = \mu_0$, as it might have been anticipated.

Now it turns out that the second integral at right hand side, say $\phi_2(t)$, is also a c.f. To see that, we must check that $\phi_2(0) = 1$, that $\phi_2$ is continuous at $t=0$ and also that the function $\phi_2$ is positive definite (p.d.). The first requirement is obvious, the second is proved by dominated convergence. Now turn to the p.d. requirement: if the prior distribution written as $p(\Sigma)\text{d}\Sigma$ is a Dirac mass, then $\phi_2$ is p.d. because $\phi_2$ is then the c.f. of a normal distribution. If the prior is a discrete mixture of Dirac masses, this is true as well since $\phi_2$ then is the c.f. of a mixture of normals. By a continuity argument, we see that $\phi_2$ is p.d.

Now enterslet us use the powerful Lévy-Cramér theorem which tells that both functions $\phi_j$ for $j=1$, $2$ must take the form $\exp\{a_j i t - b_jt^2 /2 \}$ with $a_j$ real and $b_j \geq 0$. So $\mu$ must be normal (possibly degenerate) with mean $a_1 = \mu_0$. By simple algebra we then have $$ \exp\{ -(\Sigma_0 - b_1) t^2 /2 \} = \int_0^\infty \exp\{ - \Sigma t^2 /2\} p(\Sigma) \, \text{d} \Sigma $$ which holds for any real $t$. Since any non-negative real writes as $t^2/2$, we see that the Laplace transform of the prior of $\Sigma$ must be equal to that of the Dirac mass at $\Sigma_0 - b_1$ and we are done.

Assume that $\mu$ and $\Sigma$ are a priori independent and that $y$ has a normal margin with mean $\mu_0$ and variance $\Sigma_0$. I will prove that then the variance $\Sigma$ must be constant, and the mean $\mu$ must have a normal prior (possibly degenerate).

I will stick to the one-dimensional case for simplicity, using the characteristic function (c.f.) of $y$, i.e. $\phi_y(t) := \mathbb{E}[e^{yit}]$. We know that $\phi_y(t) = \exp\{\mu_0 it - \Sigma_0 t^2 /2$} and a similar formula holds for the distribution of $y$ conditional on $\mu$ and $\Sigma$, which is normal by assumption. So for any real $t$ $$ \mathbb{E}[e^{yit}] = \int \mathbb{E}\left[e^{yit} \, \vert \,\mu,\,\Sigma\right]\, p(\mu) p(\Sigma) \,\text{d}\mu \text{d} \Sigma = \int \exp\left\{ \mu it - \Sigma t^2/2 \right\} \,p(\mu) p(\Sigma) \,\text{d}\mu \text{d}\Sigma, $$ and by rearranging the integral, we must have $$ \exp\left\{ \mu_0 it - \Sigma_0 t^2 /2 \right\} = \left[\int \exp\left\{ \mu it \right\} p(\mu) \,\text{d}\mu \right] \left[\int \exp\left\{ -\Sigma t^2/2\right\} p(\Sigma) \,\text{d}\Sigma \right]. $$ The assumptions needed for such a rearrangement are easily checked.

The first integral at right hand side, say $\phi_1(t)$, is the c.f. of $\mu$. Note that since $\phi_1(t) e^{-\mu_0 it}$ is found to be real, we see that the distribution of $\mu$ is symmetric w.r.t. $\mu_0$, and hence that $\mathbb{E}[\mu] = \mu_0$, as it might have been anticipated.

Now it turns out that the second integral at right hand side, say $\phi_2(t)$, is also a c.f. To see that, we must check that $\phi_2(0) = 1$, that $\phi_2$ is continuous at $t=0$ and also that the function $\phi_2$ is positive definite (p.d.). The first requirement is obvious, the second is proved by dominated convergence. Now turn to the p.d. requirement: if the prior distribution written as $p(\Sigma)\text{d}\Sigma$ is a Dirac mass, then $\phi_2$ is p.d. because $\phi_2$ is then the c.f. of a normal distribution. If the prior is a discrete mixture of Dirac masses, this is true as well since $\phi_2$ then is the c.f. of a mixture of normals. By a continuity argument, we see that $\phi_2$ is p.d.

Now enters the powerful Lévy-Cramér theorem which tells that both functions $\phi_j$ for $j=1$, $2$ must take the form $\exp\{a_j i t - b_jt^2 /2 \}$ with $a_j$ real and $b_j \geq 0$. So $\mu$ must be normal (possibly degenerate) with mean $a_1 = \mu_0$. By simple algebra we then have $$ \exp\{ -(\Sigma_0 - b_1) t^2 /2 \} = \int_0^\infty \exp\{ - \Sigma t^2 /2\} p(\Sigma) \, \text{d} \Sigma $$ which holds for any real $t$. Since any non-negative real writes as $t^2/2$, we see that the Laplace transform of the prior of $\Sigma$ must be equal to that of the Dirac mass at $\Sigma_0 - b_1$ and we are done.

Assume that $\mu$ and $\Sigma$ are a priori independent and that $y$ has a normal margin with mean $\mu_0$ and variance $\Sigma_0$. I will prove that then the variance $\Sigma$ must be constant, and the mean $\mu$ must have a normal prior (possibly degenerate).

I will stick to the one-dimensional case for simplicity, using the characteristic function (c.f.) of $y$, i.e. $\phi_y(t) := \mathbb{E}[e^{yit}]$. We know that $\phi_y(t) = \exp\{\mu_0 it - \Sigma_0 t^2 /2$} and a similar formula holds for the distribution of $y$ conditional on $\mu$ and $\Sigma$, which is normal by assumption. So for any real $t$ $$ \mathbb{E}[e^{yit}] = \int \mathbb{E}\left[e^{yit} \, \vert \,\mu,\,\Sigma\right]\, p(\mu) p(\Sigma) \,\text{d}\mu \text{d} \Sigma = \int \exp\left\{ \mu it - \Sigma t^2/2 \right\} \,p(\mu) p(\Sigma) \,\text{d}\mu \text{d}\Sigma, $$ and by rearranging the integral, we must have $$ \exp\left\{ \mu_0 it - \Sigma_0 t^2 /2 \right\} = \left[\int \exp\left\{ \mu it \right\} p(\mu) \,\text{d}\mu \right] \left[\int \exp\left\{ -\Sigma t^2/2\right\} p(\Sigma) \,\text{d}\Sigma \right]. $$ The assumptions needed for such a rearrangement are easily checked.

The first integral at right hand side, say $\phi_1(t)$, is the c.f. of $\mu$. Note that since $\phi_1(t) e^{-\mu_0 it}$ is found to be real, we see that the distribution of $\mu$ is symmetric w.r.t. $\mu_0$, and hence that $\mathbb{E}[\mu] = \mu_0$, as it might have been anticipated.

Now it turns out that the second integral at right hand side, say $\phi_2(t)$, is also a c.f. To see that, we must check that $\phi_2(0) = 1$, that $\phi_2$ is continuous at $t=0$ and also that the function $\phi_2$ is positive definite (p.d.). The first requirement is obvious, the second is proved by dominated convergence. Now turn to the p.d. requirement: if the prior distribution written as $p(\Sigma)\text{d}\Sigma$ is a Dirac mass, then $\phi_2$ is p.d. because $\phi_2$ is then the c.f. of a normal distribution. If the prior is a discrete mixture of Dirac masses, this is true as well since $\phi_2$ then is the c.f. of a mixture of normals. By a continuity argument, we see that $\phi_2$ is p.d.

Now let us use the powerful Lévy-Cramér theorem which tells that both functions $\phi_j$ for $j=1$, $2$ must take the form $\exp\{a_j i t - b_jt^2 /2 \}$ with $a_j$ real and $b_j \geq 0$. So $\mu$ must be normal (possibly degenerate) with mean $a_1 = \mu_0$. By simple algebra we then have $$ \exp\{ -(\Sigma_0 - b_1) t^2 /2 \} = \int_0^\infty \exp\{ - \Sigma t^2 /2\} p(\Sigma) \, \text{d} \Sigma $$ which holds for any real $t$. Since any non-negative real writes as $t^2/2$, we see that the Laplace transform of the prior of $\Sigma$ must be equal to that of the Dirac mass at $\Sigma_0 - b_1$ and we are done.

Source Link
Yves
  • 5.7k
  • 1
  • 23
  • 38

Assume that $\mu$ and $\Sigma$ are a priori independent and that $y$ has a normal margin with mean $\mu_0$ and variance $\Sigma_0$. I will prove that then the variance $\Sigma$ must be constant, and the mean $\mu$ must have a normal prior (possibly degenerate).

I will stick to the one-dimensional case for simplicity, using the characteristic function (c.f.) of $y$, i.e. $\phi_y(t) := \mathbb{E}[e^{yit}]$. We know that $\phi_y(t) = \exp\{\mu_0 it - \Sigma_0 t^2 /2$} and a similar formula holds for the distribution of $y$ conditional on $\mu$ and $\Sigma$, which is normal by assumption. So for any real $t$ $$ \mathbb{E}[e^{yit}] = \int \mathbb{E}\left[e^{yit} \, \vert \,\mu,\,\Sigma\right]\, p(\mu) p(\Sigma) \,\text{d}\mu \text{d} \Sigma = \int \exp\left\{ \mu it - \Sigma t^2/2 \right\} \,p(\mu) p(\Sigma) \,\text{d}\mu \text{d}\Sigma, $$ and by rearranging the integral, we must have $$ \exp\left\{ \mu_0 it - \Sigma_0 t^2 /2 \right\} = \left[\int \exp\left\{ \mu it \right\} p(\mu) \,\text{d}\mu \right] \left[\int \exp\left\{ -\Sigma t^2/2\right\} p(\Sigma) \,\text{d}\Sigma \right]. $$ The assumptions needed for such a rearrangement are easily checked.

The first integral at right hand side, say $\phi_1(t)$, is the c.f. of $\mu$. Note that since $\phi_1(t) e^{-\mu_0 it}$ is found to be real, we see that the distribution of $\mu$ is symmetric w.r.t. $\mu_0$, and hence that $\mathbb{E}[\mu] = \mu_0$, as it might have been anticipated.

Now it turns out that the second integral at right hand side, say $\phi_2(t)$, is also a c.f. To see that, we must check that $\phi_2(0) = 1$, that $\phi_2$ is continuous at $t=0$ and also that the function $\phi_2$ is positive definite (p.d.). The first requirement is obvious, the second is proved by dominated convergence. Now turn to the p.d. requirement: if the prior distribution written as $p(\Sigma)\text{d}\Sigma$ is a Dirac mass, then $\phi_2$ is p.d. because $\phi_2$ is then the c.f. of a normal distribution. If the prior is a discrete mixture of Dirac masses, this is true as well since $\phi_2$ then is the c.f. of a mixture of normals. By a continuity argument, we see that $\phi_2$ is p.d.

Now enters the powerful Lévy-Cramér theorem which tells that both functions $\phi_j$ for $j=1$, $2$ must take the form $\exp\{a_j i t - b_jt^2 /2 \}$ with $a_j$ real and $b_j \geq 0$. So $\mu$ must be normal (possibly degenerate) with mean $a_1 = \mu_0$. By simple algebra we then have $$ \exp\{ -(\Sigma_0 - b_1) t^2 /2 \} = \int_0^\infty \exp\{ - \Sigma t^2 /2\} p(\Sigma) \, \text{d} \Sigma $$ which holds for any real $t$. Since any non-negative real writes as $t^2/2$, we see that the Laplace transform of the prior of $\Sigma$ must be equal to that of the Dirac mass at $\Sigma_0 - b_1$ and we are done.