Computing or sampling from a posterior with samples observed through a dimensional reduction transformation

Question

Let $\boldsymbol \theta$ be a vector of parameters, with a known prior $\pi(\boldsymbol \theta)$. Let $\boldsymbol x_1,...,\boldsymbol x_n$ be i.i.d. samples with $\boldsymbol x|\boldsymbol \theta$ known. The problem is that we do not observe $\boldsymbol x_i$, but $\boldsymbol g(\boldsymbol x_i)$. Here $\boldsymbol g$ is a dimensional reduction transformation. How to numerically compute the posterior $\pi(\boldsymbol \theta | \boldsymbol g(\boldsymbol x_1),...,g(\boldsymbol x_n))$ or sampling from it efficiently?

A simple example is here. For a rectangle, the width $w\sim {\rm Uniform}(0, a)$ and the length $l\sim {\rm Uniform}(0,b)$, and $(a,b)$ follows a known prior. But we cannot observe $(w,l)$. Instead, we can observe the area $s = w\cdot l$. How to compute the posterior $\pi(a,b|s)$ numerically or sampling from it?

Answer 1

If the transform$$g:\rm X\longrightarrow \rm G$$is too complex to analytically determine the density of the distribution of $g(X)$, a plain solution is to reconstruct $X$ conditional on $g(X)$ and $\theta$ via a Gibbs sampler:

At iteration $t$,

1. generate $X_1^t,\ldots,X_n^t$ conditional on $g(X_1),\ldots,g(X_n)$ and $\theta^{t-1}$
2. generate $\theta^t$ conditional on $X_1^t,\ldots,X_n^t$

This is for instance what we did in our recent Insufficient Gibbs sampling paper for the specific case of $g(x_1,\ldots,x_n)$ being observed and corresponding to quantile and/or MAD statistics.

In the event the (constrained) generation of $X_1^t,\ldots,X_n^t$ is too complex, as suggested by Guillaume Dehaene, an ABC algorithm can take over by simulating

1. $\theta^t$ from the prior $\pi(\cdot)$
2. $X_1^t,\ldots,X_n^t$ from the sampling distribution $f(\cdot|\theta^t)$

and only keeping the $\theta^t$'s for which $g(X_1^t),\ldots,g(X_n^t)$ is close enough to the observed $g(X_1^0),\ldots,g(X_n^0)$