0
$\begingroup$

Setup

Suppose we have an exponential family model $\{P_{\theta} : \theta \in \Theta\}$. Let the density function of a random variable $X$ and the prior on $\theta$ have following forms:

$$ \begin{align} p_{\theta}(x) &= h_1(x) \exp\left\{\theta^{\top} u(x) - a(\theta)\right\}, \tag{1} \\ p_{\chi, \nu}(\theta) &= h_2(\chi, \nu) \exp\left\{\theta^{\top} \chi - \nu a(\theta) \right\}. \tag{2} \end{align} $$

Here, $h_1$ and $h_2$ are base measures, $u(x)$ are sufficient statistics, and $a(\theta)$ is the log-normalizer for $p_{\theta}(x)$. It's a standard result that the posterior is

$$ \pi_{\chi, \nu}(\theta) \prod_{n=1}^N p_{\theta}(x) \propto \exp\left\{ \theta^{\top}\left(\chi + \sum_{n=1}^N u(x_n) \right) - \left(\nu + N \right) a(\theta) \right\} \propto \pi_{\chi_N, \nu_N}(\theta), \tag{3} $$

where $\chi_N = \chi + \sum u(x_n)$ and $\nu_N = \nu + N$. This demonstrates conjugacy.

My question

I've seen the general formulation of equations $1$ and $2$ many times, but I couldn't tell you concretely what $h_2$, $\chi$, and $\nu$ are in specific cases. For example, consider this Gaussian model:

$$ \begin{align} X &\sim \mathcal{N}(\theta, \sigma_x^2), \tag{4} \\ \theta &\sim \mathcal{N}(\mu_0, \sigma_0^2), \tag{5} \end{align} $$

I can easily derive various quantities, e.g. the posterior predictive directly, using basic properties of the Gaussian. But I'm not sure how to do this using a general formula for the posterior predictive (see Wikipedia). While both densities (equations $4$ and $5$) are Gaussian, they have slightly different functional forms in equation $1$ and $2$. I can compute that $h_1(x) = 1 / \sqrt{2 \pi}$, the normalizer, for example, but then what is $h_2(\chi, \nu)$?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.