Gaussian distribution with error in both parameters: trying to create a Gaussian distribution with no error in either parameter

Question

I am trying to create a Gaussian distribution where the parameters are not precisely known. I have mean $\mu$ and standard deviation of the mean $\sigma_\mu$ (modeled as an uncorrelated Gaussian), and standard deviation $\sigma$ and standard deviation of the standard deviation $\sigma_\sigma$ (I believe this is commonly modeled as Gaussian as well, but I'm not certain. It is also uncorrelated.) From this, I would imagine that it is possible to create a distribution that captures all of these parameters at once (with no error in the parameters).

I have the feeling that the target distribution is Gaussian. Putting the $\mu$ term and $\sigma$ term in will result in:

$X \sim \mathcal{N}(\mu, \sigma^2)$

Then adding in the standard deviation of $\sigma_\mu$ (a distribution with mean 0) should result in this, by means of adding two Gaussians, which is a Gaussian:

$X \sim \mathcal{N}(\mu + 0, \sigma^2+\sigma_\mu^2)$

At this point I have to put in the standard deviation of the standard deviation $\sigma_\sigma$ and I am stuck. I think there is a way to put $\sigma_\sigma$ into the variance spot but I am not sure how.

Answer 1

To analyse the distribution properly you cannot start with the assumption that the result will be Gaussian. If $$X|\theta,\tau\sim\mathcal N(\theta,\tau^2)$$ and $$\theta\sim\mathcal N(\mu,\sigma^2_\theta)\qquad\tau\sim\mathcal N(\sigma,\omega^2_\tau)$$ then one can write $$X=\theta+\tau\xi=\mu+\sigma_\theta\epsilon_\theta+(\sigma+\omega_{\tau}\epsilon_\tau)\xi$$ where $$\epsilon_\theta,\epsilon_\tau,\xi\sim\mathcal N(0,1)$$independently. From this representation, one can see that the resulting distribution of $X$ is not Normal due to the product $\epsilon_\tau\times\xi$ of standard Normal variates. Since by integrating out $\theta$ $$X|\tau\sim\mathcal N(\mu,\sigma^2_\theta+\tau^2)$$the marginal density of $X$ is given by $$ f(x)=\int_{-\infty}^\infty \frac{(\sigma^2_\theta+\tau^2)^{-1/2}}{\sqrt{2\pi}} \exp\left\{-(x-\mu)^2/2\tau^2\right\}\frac{1}{\sqrt{2\pi}\omega_\tau}\exp\{-(\tau-\sigma)^2/2\omega^2_\tau\}\,\text d\tau$$ which is not particularly manageable (i.e., does not return a closed form expression).

Note that the Normal assumption on $\tau$ is rather unusual because usually $\tau$ is assumed to be positive. A more standard approach (in a Bayesian perspective) is to assume that $\tau^{-2}$ is distributed from a Gamma distribution$$\tau^{-2}\sim\mathcal Ga(\alpha,\beta)$$ Then$$X|\tau\sim\mathcal N(\mu,\sigma^2_\theta+\tau^2)$$and the marginal density of $X$ is given by $$ f(x)=\int_0^\infty \frac{(\sigma^2_\theta+\iota^{-1})^{-1/2}}{\sqrt{2\pi}} \exp\left\{-(x-\mu)^2\iota/2\right\}\iota^{\alpha-1}\exp\{-\beta\iota\}\,\text d\iota$$ for which there is no closed-form expression (unless using special functions).

Another Bayesian modelling is to assume that $\theta$ and $\tau$ are independent, with $$\theta|\tau\sim\mathcal N(\mu,\rho^2\tau^2)\qquad\tau^{-2}\sim\mathcal Ga(\alpha,\beta)$$ in which case the marginal distribution of $X$ is a Student's $t$ distribution (cf. Bayesian textbooks).

Answer 2

You are ignoring the covariance between this error and the original normal variable. Since you didn't specify the structure of the estimator or of the error, it does not follow that the resulting scale parameter of the normal will be $\sigma^2 + \sigma_\sigma^2$.