2
$\begingroup$

Suppose $p(X_j | X_1,\ldots,X_{j-1})$ comes from some known exponential family for every $j=1,\ldots,k$. Does it follow that the joint distribution comes from some (possibly different) exponential family?

An obvious example where this is true is Gaussian RVs. Are there conditions under which this generalizes? (Note that unlike the Gaussian case, I am not requiring that the joint belong to the same exponential family; this is not even really well-defined anyway.)

$\endgroup$
1
  • $\begingroup$ Since the conditional probability is a function of the other $X_i,$ you seem to be asserting that a sequence of functions of zero, one, ..., through $k-1$ variables "come from some known exponential family." Could you state more specifically what that might mean? $\endgroup$
    – whuber
    Apr 22, 2022 at 17:53

1 Answer 1

2
$\begingroup$

Let us assume that the conditional density writes as $(j=1,\ldots,d)$ $$p_j(x_j|x_{-j})= h_j(x_j)\exp\{A_j(x_j)^\text TB_j(\theta_j,x_{-j})-C_j(\theta,x_{-j})\}$$ where the various functions are arbitrary (but such that the density is integrable with mass one wrt the dominating measure). Then by the Hammersley-Clifford theorem, the joint density can be represented as the (Gibbs) product of conditionals $$p(x)\propto\prod_{j=1}^d \dfrac{p_j(x_j|x^0_{<j},x_{>j})}{p_j(x^0_j|x^0_{<j},x_{>j})}\tag{1}$$ where $x⁰$ is an arbitrary (but possible) value. This identity then implies that \begin{align}p_j(x_j|x_{-j})&\propto \prod_{j=1}^d h_j(x_j) \exp\{A_j(x_j)^\text TB_j(\theta_j,x^0_{<j},x_{>j})-C_j(\theta,x^0_{<j},x_{>j})\\&\qquad-A_j(x^0_j)^\text TB_j(\theta_j,x^0_{<j},x_{>j})+C_j(\theta,x^0_{<j},x_{>j})\}\\ &\propto H(\mathbf x)\exp\left\{\sum_{j=1}^d\left[A_j(x_j)^\text TB_j(\theta_j,x^0_{<j},x_{>j})-A_j(x^0_j)^\text TB_j(\theta_j,x^0_{<j},x_{>j})\right]\right\} \end{align} which does not factorise as an exponential family in general, since the functions $B_j$ that appear within the exponential do not necessarily separate as functions of $\theta$ times functions of $\mathbf x$.

In the somewhat different case mentioned in the question, that is when $$p_j(x_j|x_{<j})= h_j(x_j)\exp\{A_j(x_j)^\text TB_j(\theta_j,x_{<j})-C_j(\theta,x_{<j})\}$$ the representation (1) still holds and again does not imply a joint exponential family distribution. If one uses instead the product rule $$p(\mathbf x)\propto\prod_{j=1}^d p_j(x_j|x_{<j})\tag{2}$$ the joint density writes as $$p(\mathbf x)\propto H(\mathbf x)\exp\left\{\sum_{j=1}^d\left[A_j(x_j)^\text TB_j(\theta_j,x_{<j})-C_j(\theta_j,x_{<j})\right]\right\}$$ and again there is no generic decomposition as an exponential family density.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.