# Distribution of the sufficient statistic in the exponential family?

Suppose $$\boldsymbol X$$ belongs to the exponential family,

$$f_X\!\left(\,\mathbf{x} ; \boldsymbol \theta\,\right) = h(\mathbf{x}) \, \exp\!\Big(\,\boldsymbol\eta({\boldsymbol \theta}) \cdot \mathbf{T}(\mathbf{x}) - A({\boldsymbol \theta})\,\Big)$$

with $$\mathbf{T}(\mathbf{x})$$ the sufficient statistic.

Is there a way to factorize $$f_X\!\left(\,\mathbf{x};\boldsymbol \theta\,\right)$$ such that

$$f_X\!\left(\,\mathbf{x};\boldsymbol \theta\,\right) = f_T\!\left(\,\mathbf{T};\boldsymbol \theta\,\right) f_{X|T}\!\left(\,\mathbf{x} | \mathbf{T}\right)$$

where $$f_{X|T}\!\left(\,\mathbf{x} | \mathbf{T}\right)$$ is the distribution of $$\boldsymbol X$$ conditional on $$\mathbf{T}$$, and $$f_T\!\left(\,\mathbf{T};\boldsymbol \theta\,\right)$$ is the marginal distribution of $$\mathbf{T}$$?

In a discrete setting, when the sampling space $$\mathfrak X^n$$ (assuming an iid sample of size $$n$$ from an exponential family) is finite or countable, the conditional distribution of $$\mathbb P(\mathbf X_n=\mathbf x_n;\theta\vert T_n(\mathbf X_n)=t_n)= \dfrac{\mathbb P(\mathbf X_n=\mathbf x_n;\theta)}{\mathbb P(T_n(\mathbf X_n)=t_n;\theta)}$$ Hence, using the counting measures on $$\mathfrak X^n$$ and $$T(\mathfrak X^n)$$, respectively, $$f(\mathbf x_n;\theta) = f(\mathbf x_n\vert T_n(\mathbf X_n)=t_n) \times f(t_n;\theta)$$ In the continuous case, the result is somehow a tautology since the choice of the (three) dominating measures is open. To start, we can wlog assume that$$T_n(\mathbf x_n)=\sum_{i=1}^n x_i$$ and that the density with respect to a measure $$\lambda^{\otimes n}$$ on $$\mathfrak X^n$$ is $$f(\mathbf x_n;\theta) = \prod_{i=1}^n h(x_i) \times e^{\theta\cdot T_n(\mathbf x_n) - n A(\theta)}$$ Changing the measure into$$\text d\mu(x) \stackrel{\text{def}}{=} h(x)\,\text d\mu(x)$$leads to the density with respect to the measure $$\mu^{\otimes n}$$ on $$\mathfrak X^n$$ $$f(\mathbf x_n;\theta) = e^{\theta\cdot T_n(\mathbf x_n) - n A(\theta)}$$ The joint density of $$(T_n(\mathbf X),X_2,\ldots,X_n)$$ wrt the measure $$\mu_{T_n}\otimes\mu^{\otimes n-1}$$ on $$T_n(\mathfrak X^n)\times\mathfrak X^{n-1}$$ is then $$f(t_n,x_2,\ldots,x_n;\theta) = e^{\theta\cdot t_n - n A(\theta)}\tag{1}$$ since the Jacobian is equal to one. The marginal density of $$T_n(\mathbf X)$$ wrt $$\mu_{T_n}$$ is thus $$f(t_n;\theta) = e^{\theta\cdot t_n - n A(\theta)}\times \tilde h(t_n)$$ where the last term $$\tilde h(t_n)$$ is the mass of $$T_n(t_n-\sum_{i\ge 2} x_i,x_2,\ldots,x_n)=t_n\tag{2}$$ under $$\mu^{\otimes n-1}$$. Changing the measure $$\mu_{T_n}$$ to the new measure $$\xi_{T_n}$$ defined by $$\text d\xi_{T_n} (t)= \tilde h(t)\text d\mu_{T_n}(t)$$ leads to the following density of $$T_n(\mathbf X)$$ wrt $$\xi_{T_n}$$: $$f(t_n;\theta) = e^{\theta\cdot t_n - n A(\theta)}$$ If one defines $$\zeta_{t_n}$$ as the mass one uniform measure over the slice $$\left\{(x_1,x_2,\ldots,x_n); \ T_n(x_1,x_2,\ldots,x_n)=\sum_{i=1}^n x_i=t_n\right\},$$ the conditional density of $$\mathbf X$$ conditional on $$T_n(\mathbf X)$$ is one and $$f_X\left(\,\mathbf{x}_n;\theta\,\right) = f_{T_n}\!\left(\,t_n; \theta\,\right) f_{X|T=t_n}\!\left(\,\mathbf{x}_n | T_n=t_n\right)$$ when the densities are taken wrt $$\mu^{\otimes n-1}$$, $$\xi_{T_n}$$, and $$\zeta_{t_n}$$, respectively.

(Note: Using $$f$$ for different densities is an abuse of notation, but avoids an explosion in said notation.)

To get $$f_{T}\!\left(\,\mathbf{T} ; \boldsymbol\theta\right)$$, we take the full density and integrate $$\mathbf{x}$$ out:

\begin{align}f_{T}\!\left(\,\mathbf{T} ; \boldsymbol\theta\right) &= \int_{{\bf x} : {\bf T(x)} = {\bf T}} f_X\!\left(\,\mathbf{x} ; \boldsymbol \theta\,\right) d {\bf x} \\ &= \exp\!\Big(\,\boldsymbol\eta({\boldsymbol \theta}) \ \mathbf{T} - A({\boldsymbol \theta})\,\Big) \int_{{\bf x} : {\bf T(x)} = {\bf T}} h(\mathbf{x}) d {\bf x} \\ &= \exp\!\Big(\,\boldsymbol\eta({\boldsymbol \theta}) \ \mathbf{T} - A({\boldsymbol \theta})\,\Big) h_T({\bf T}) \end{align} Where $$h_T({\bf T}) = \int_{{\bf x} : {\bf T(x)} = {\bf T}} h(\mathbf{x}) d {\bf x}$$

Consequentially, we have

$$f_{X|T}\!\left(\,\mathbf{x} | \mathbf{T}\right) = \frac{ h({\bf x}) }{h_T({\bf T}({\bf x}))}$$

• The integral$$\int_{{\bf x} : {\bf T(x)} = {\bf T}} f_X\!\left(\,\mathbf{x} ; \boldsymbol \theta\,\right) \text d {\bf x}$$is not defined when $\text dx$ denotes the Lebesgue measure. – Xi'an Dec 9 '20 at 10:14