4
$\begingroup$

Giving the following equations $$ \mu_n = \frac{\kappa_0 \mu_0 + n \overline{x}}{\kappa_0 + n}, \\ \kappa_n = \kappa_0 + n, \\ \alpha_n = \alpha_0 + n/2, \\ \beta_n = \beta_0 + \frac{1}{2} \sum\limits_{i=1}^n (x_i - \overline{x})^2 + \frac{\kappa_0 n (\overline{x} - \mu_0)^2}{2(\kappa_0 + n)}, $$ where $\overline{x} = \frac{\sum_{i=1}^n x_i }{n}$, i.e., the mean of data. Actually, the above equations are the parameters of the posterior of normal normal-gamma conjugacy (see Equation (85-89) in this paper for details).

Here, I want to prove: $$ \mu_{n+m} = \frac{\kappa_n \mu_n + m \overline{x}}{\kappa_n + m}, \\ \kappa_{n+m} = \kappa_n + m, \\ \alpha_{n+m} = \alpha_n + m/2, \\ \beta_{n+m} = \beta_n + \frac{1}{2} \sum\limits_{i=n+1}^{n+m} (x_i - \overline{x})^2 + \frac{\kappa_n m (\overline{x} - \mu_n)^2}{2(\kappa_n + m)}, $$ where $\overline{x} = \frac{\sum_{i=n+1}^{n+m} \quad x_i}{m}$, i.e., the mean of new observations. The above equations are appeared in Equation (101-104) of this paper, in which the $m$ is set as 1.

=============================================================

The following effort I have tried $$ \kappa_{n+m} = \kappa_0 + n + m = \kappa_{n} + m \\ \alpha_{n+m} = \alpha_0 + \frac{n+m}{2} = \alpha_n + \frac{m}{2} \\ \mu_{n+m} = \frac{\kappa_0 \mu_0 + \sum_{i=1}^{n+m} x_i }{\kappa_0 + n+ m} = \frac{\kappa_0 \mu_0 + \sum_{i=1}^n x_i + \sum_{i=n+1}^{n+m} x_i }{\kappa_n + m} = \frac{(\kappa_0 + n)\frac{\kappa_0 \mu_0 + \sum_{i=1}^n x_i }{\kappa_0 + n} + \sum_{i=n+1}^{n+m} x_i }{\kappa_n + m} = \frac{\kappa_n \mu_n + m\overline{x}}{\kappa_n + m} $$ where $\overline{x} = \frac{\sum_{i=n+1}^{n+m} \quad x_i}{m}$. But I fail to prove the last equation: $$ \beta_{n+m} = \beta_n + \frac{1}{2} \sum\limits_{i=n+1}^{n+m} (x_i - \overline{x})^2 + \frac{\kappa_n m (\overline{x} - \mu_n)^2}{2(\kappa_n + m)}. $$ So, could anybody give me some suggestions?

$\endgroup$
3
$\begingroup$

A direct and elegant way to show these identities is to exploit that the posterior distribution $\pi_{n+m}(\theta|\mathfrak D_{n+m})$ associated with the prior $\pi_0$ and the $n+m$ observations $(x_1,\ldots,x_{n+m})$ is equal to the posterior distribution associated with the "prior" $\pi_n(\theta|\mathfrak D_{n})$ and the $m$ observations $(x_{n+1},\ldots,x_{n+m})$. When the prior is conjugate, e.g., Normal-Gamma, the intermediary posterior $\pi_n(\theta|\mathfrak D_{n})$ is also conjugate and the update of the hyperparameters is immediate.

A more pedestrian and dreary approach is to expand the expression of $\beta_{n+m}$. For the last update equation, let us introduce the notations $\overline{x}_{1:n+m}$, $\overline{x}_{1:n}$ and $\overline{x}_{(n+1):(n+m)}$ to distinguish between the three averages. Then \begin{align} \beta_{n+m} &= \beta_0 + \frac{1}{2} \sum\limits_{i=1}^{n+m} (x_i - \overline{x}_{1:n+m})^2 + \frac{\kappa_0 (n+m) (\overline{x}_{1:n+m} - \mu_0)^2}{2(\kappa_0 + n+m)}\\ &= \beta_0 + \frac{1}{2} \sum\limits_{i=1}^{n} (x_i - \overline{x}_{1:n}+\overline{x}_{1:n}-\overline{x}_{1:n+m})^2\\ &\quad + \frac{1}{2} \sum\limits_{i=n+1}^{n+m} (x_i - \overline{x}_{(n+1):(n+m)}+ \overline{x}_{(n+1):(n+m)}- \overline{x}_{1:n+m})^2\\ &\quad + \frac{\kappa_0 (n\overline{x}_{1:n}+m\overline{x}_{(n+1):(n+m)} - (n+m)\mu_0)^2}{2(\kappa_0 + n+m)(n+m)}\\ &= \beta_0 + \frac{1}{2} \sum\limits_{i=1}^{n} (x_i - \overline{x}_{1:n})^2+\frac{1}{2}n(\overline{x}_{1:n}-\overline{x}_{1:n+m})^2\\ &\quad + \frac{1}{2} \sum\limits_{i=n+1}^{n+m} (x_i - \overline{x}_{(n+1):(n+m)})^2+ \frac{1}{2}m(\overline{x}_{(n+1):(n+m)}- \overline{x}_{1:n+m})^2\\ &\quad + \frac{\kappa_0 (n\overline{x}_{1:n} - n\mu_0)^2}{2(\kappa_0 + n+m)(n+m)}+ \frac{\kappa_0 (m\overline{x}_{(n+1):(n+m)} - m\mu_0)^2}{2(\kappa_0 + n+m)(n+m)}\\ &\quad + \frac{2\kappa_0 (n\overline{x}_{1:n}-n\mu_0)(m\overline{x}_{(n+1):(n+m)} - m\mu_0)}{2(\kappa_0 + n+m)(n+m)}\\ &= \beta_n + \frac{1}{2} \sum\limits_{i=n+1}^{n+m} (x_i - \overline{x}_{(n+1):(n+m)})^2 + R_{n,m}\end{align} with \begin{align} R_{n,m}&=\frac{1}{2}n(\overline{x}_{1:n}-\overline{x}_{1:n+m})^2+ \frac{1}{2}m(\overline{x}_{(n+1):(n+m)}- \overline{x}_{1:n+m})^2\\ &\quad + \frac{\kappa_0 n^2(\overline{x}_{1:n} - \mu_0)^2}{2(\kappa_0 + n+m)(n+m)}+ \frac{\kappa_0 m^2(\overline{x}_{(n+1):(n+m)} - \mu_0)^2}{2(\kappa_0 + n+m)(n+m)}\\ &\quad + \frac{2\kappa_0 nm(\overline{x}_{1:n}-\mu_0)(\overline{x}_{(n+1):(n+m)} - \mu_0)}{2(\kappa_0 + n+m)(n+m)}- \frac{\kappa_0 n(\overline{x}_{1:n} - \mu_0)^2}{2(\kappa_0 + n)}\\ &=\frac{nm}{2(n+m)}(\overline{x}_{1:n}-\overline{x}_{(n+1):(n+m)})^2\\ &\quad + \frac{\kappa_0 n^2(\overline{x}_{1:n} - \mu_0)^2}{2(\kappa_0 + n+m)(n+m)}+ \frac{\kappa_0 m^2(\overline{x}_{(n+1):(n+m)} - \mu_0)^2}{2(\kappa_0 + n+m)(n+m)}\\ &\quad + \frac{2\kappa_0 nm(\overline{x}_{1:n}-\mu_0)(\overline{x}_{(n+1):(n+m)} - \mu_0)}{2(\kappa_0 + n+m)(n+m)} - \frac{\kappa_0 n(\overline{x}_{1:n} - \mu_0)^2}{2(\kappa_0 + n)}\\ &= \overline{x}_{(n+1):(n+m)}^2 \left\{ \frac{nm}{2(n+m)} + \frac{\kappa_0 m^2}{2(\kappa_0 + n+m)(n+m)}\right\}\\ &\quad + \overline{x}_{(n+1):(n+m)} \left\{ - \frac{2nm}{2(n+m)}\overline{x}_{1:n} - \frac{2\kappa_0 m^2\mu_0}{2(\kappa_0 + n+m)(n+m)}\right.\\ &\qquad\qquad\qquad\left. + \frac{2\kappa_0 nm(\overline{x}_{1:n}-\mu_0)}{2(\kappa_0 + n+m)(n+m)} \right\}\\ &\quad+ \frac{nm}{2(n+m)}\overline{x}_{1:n}^2+ \frac{\kappa_0 n^2(\overline{x}_{1:n} - \mu_0)^2}{2(\kappa_0 + n+m)(n+m)} - \frac{2\kappa_0 nm(\overline{x}_{1:n}-\mu_0)\mu_0}{2(\kappa_0 + n+m)(n+m)}- \frac{\kappa_0 n(\overline{x}_{1:n} - \mu_0)^2}{2(\kappa_0 + n)}+ \frac{\kappa_0 m^2\mu_0^2}{2(\kappa_0 + n+m)(n+m)}\\ &=\overline{x}_{(n+1):(n+m)}^2 \frac{\kappa_nm}{2(\kappa_n +m)}\\ &\quad +\overline{x}_{(n+1):(n+m)}\overbrace{\frac{2m}{2(\kappa_n +m)(n+m)} \left\{ -[\kappa_n+m-\kappa_0]n \overline{x}_{1:n}-[n+m]\kappa_0\mu_0\right\}}^{-2\frac{\kappa_nm}{2(\kappa_n+m)}\mu_n}\\ &\quad +\frac{n\overline{x}_{1:n}^2}{2(\kappa_n+m)\kappa_n(n+m)} \overbrace{\left\{(\kappa_n+m)\kappa_nm+\kappa_0\kappa_nn-\kappa_0(\kappa_n+m)(n+m)\right\}}^{mn^2+nm^2=mn(m+n)}\\ &\quad -2\frac{n\overline{x}_{1:n}\kappa_0\mu_0}{2(\kappa_n+m)\kappa_n(n+m)}\overbrace{\left\{ n\kappa_n+m\kappa_n-(\kappa_n+m)(n+m)\right\}}^{-(n+m)m}\\ &\quad +\frac{\kappa_0\mu_0^2}{2(\kappa_n+m)\kappa_n(n+m)}\overbrace{\left\{ n^2\kappa_n+2nm\kappa_n+m^2\kappa_n-n(\kappa_n+m)(n+m)\right\}}^{\kappa_0m(n+m)}\\ &=\overline{x}_{(n+1):(n+m)}^2 \frac{\kappa_nm}{2(\kappa_n +m)} -2\overline{x}_{(n+1):(n+m)}\mu_n \frac{\kappa_nm}{2(\kappa_n +m)}+ \mu_n^2 \frac{\kappa_nm}{2(\kappa_n +m)}\\ &= \frac{\kappa_n m }{2(\kappa_n + m)} \, (\overline{x}_{(n+1):(n+m)} - \mu_n)^2 \end{align}

$\endgroup$
1
  • 1
    $\begingroup$ Thanks for your patience in answering! I saw you edit the answer so many times, and I have checked the final version, it is correct! Thank you exactly! $\endgroup$
    – jzin
    Mar 7 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.