0
$\begingroup$

How is it possible to calculate the variance $\sigma^2$ for the Normal distribution if only $\alpha$ and $\beta$ (based on data) from the Inverse-gamma distribution are available? I followed the definitions on the Wiki and found out that for a Normal distribution in case of unknown variance and known mean the cojugate distribution is the Inverse-gamma with the updates $\alpha = \alpha + \frac{1}{2}$ and $\beta = \beta + \frac{(x - \mu)^2}{2}$.

I tried $\sigma^2 = \frac{\beta^2}{(\alpha - 1)^2 (\alpha - 2)} \text{for } \alpha > 2$ but $\alpha$ becomes less than $2$ after an update, and the variance returns NaN.

Thanks for any suggestions.

$\endgroup$
  • $\begingroup$ What do you mean by "only"? You do not have data? $\endgroup$ – Tim Nov 8 '16 at 11:40
  • $\begingroup$ Edited the question and added the fact that data is used. $\endgroup$ – NumbThumb Nov 8 '16 at 12:00
  • $\begingroup$ But do you want to estimate variance of the distribution of $\sigma^2$, or do you want to estimate $\sigma^2$ itself? $\endgroup$ – Tim Nov 8 '16 at 12:14
  • $\begingroup$ I want to estimate $\sigma^2$ in order to calculate the Normal distribution using Conjugate families. In case of Gaussian with known mean and unknown variance the conjugate resembles the Inverse-gamma distribution with the already mentioned update rules (see original post). $\endgroup$ – NumbThumb Nov 8 '16 at 12:21
1
$\begingroup$

When estimating the variance of normal distribution with known mean $\mu$ using conjugate inverse gamma prior, the model is

$$ X \sim \mathrm{Normal}(\mu, \sigma^2) \\ \sigma^2 \sim \mathrm{IG}(\alpha, \beta) $$

So we assume an inverse gamma prior parametrized by $\alpha$ and $\beta$ for $\sigma^2$. When we observe some data $x_1,\dots,x_n$ we can update our prior, to get posterior distribution of $\sigma^2$ and since inverse gamma is a conjugate prior, the update has a simple closed-form solution. The updated parameters become

$$ \alpha' = \alpha + \frac{n}{2}, \qquad \beta' = \frac{\sum_{i=1}^n (x_i - \mu)^2}{2} $$

and the posterior distribution of $\sigma^2$ is inverse gamma parametrized by $\alpha'$ and $\beta'$. So by updating the $\alpha$ and $\beta$ parameters you already estimated the distribution of $\sigma^2$. If you want to obtain a point estimate of $\sigma^2$, then you can calculate mean $\tfrac{\beta'}{\alpha'-1}$ or mode $\tfrac{\beta'}{\alpha'+1}$ of it's posterior distribution. What you were trying to do, is you were trying to calculate the variance of $\sigma^2$, that is certainly not it's point estimate.

Check also Bayesian updating with conjugate priors using the closed form expressions and Bayesian updating with new data for similar questions.

$\endgroup$
  • $\begingroup$ Thanks, this sounds plausible. Will have a more detailed look in a couple of minutes and accept your answer afterwards. Maybe I will come up with some questions if I work through the similar questions. $\endgroup$ – NumbThumb Nov 8 '16 at 12:59
  • $\begingroup$ The mean or mode calculation of the point estimate of $\sigma^2$ both take the form $\frac{\beta'}{\alpha'-1}$, is this correct or just a typo? $\endgroup$ – NumbThumb Nov 8 '16 at 15:00
  • $\begingroup$ @NumbThumb they differ by using + or - sign see en.wikipedia.org/wiki/Inverse-gamma_distribution $\endgroup$ – Tim Dec 20 '16 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.