# Behaviour of the marginal in the limit for an infinite sequence of hierarchical priors

Consider the following model:

$$y \sim \text{Exponential}(\lambda_0) \\ \lambda_i | \lambda_{i+1} \sim \text{Exponential}(\lambda_i+1) \\ \text{for } i=1,2,\dots,d\\ \lambda_{d+1} = k$$ With an observed $$y=c$$.

How does the marginal likelihood given a fixed number $$d$$ of priors $$P_d(y)$$ behave as a function of $$d$$? How about $$\lim_{d\to\infty} P_d(y)$$

Can a general statement be made for arbitrary distributions?

• projecteuclid.org/euclid.bj/1080004760 investigates these sorts of questions – Cyan Oct 22 '20 at 13:13
• @Cyan oh wow there's actually a paper on this exact question. I read the abstract, this is definitely going on my steadily growing reading list! Thanks so much for sharing! – user228809 Oct 22 '20 at 22:01

The parameter of the Exponential distribution is an inverse scale parameter: $$X\sim \mathcal E(\lambda)$$ can be represented as$$X=\epsilon/\lambda\qquad\epsilon\sim\mathcal E(1)$$Therefore if $$X \sim \mathcal E(\lambda_0) \\ \lambda_i | \lambda_{i+1} \sim \mathcal E(\lambda_{i+1}) \quad i=0,1,\dots,d\\ \lambda_{d+1} = 1$$ we can write \begin{align}X &= \epsilon_0 / \lambda_0 \qquad &\epsilon_0\sim\mathcal E(1)\\ &= \lambda_1 \epsilon_0 / \epsilon_1 \qquad &\epsilon_1\sim\mathcal E(1)\\ &= \epsilon_2 \epsilon_0 / \epsilon_1 \lambda_2 \qquad &\epsilon_2\sim\mathcal E(1)\\ &\quad \vdots\\ &= \frac{\epsilon_{2\lfloor d/2 \rfloor}\cdots \epsilon_0}{\epsilon_{2\lfloor (d-1)/2 \rfloor+1}\cdots \epsilon_1}\qquad &\epsilon_d\sim\mathcal E(1) \end{align} and hence \begin{align*}X&=\frac{\exp\left\{\sum_{i=0}^{\lfloor d/2 \rfloor}\log\epsilon_{2i}\right\}}{\exp\left\{\sum_{i=0}^{\lfloor (d-1)/2 \rfloor}\log\epsilon_{2i+1}\right\}}\qquad \epsilon_i\sim\mathcal E(1)\\ &=\frac{\exp\left\{\sum_{i=0}^{\lfloor d/2 \rfloor}\log\{-\log\upsilon_{2i}\}\right\}}{\exp\left\{\sum_{i=0}^{\lfloor (d-1)/2 \rfloor}\log\{-\log\upsilon_{2i+1}\}\right\}} \qquad \upsilon_i\sim\mathcal U(0,1)\end{align*} appears as the ratio of two independent random variables both converging a.s. to zero with $$d$$, hence having no limiting distribution. (This property generalises to other scale distributions, obviously, provided $$\mathbb E[\sqrt{\epsilon_i}]<1$$.)
Considering the general case $$\lambda_d\sim\mathcal E(k)$$ simply means alternatively multiplying and dividing by $$k$$ the above, hence preventing any form of convergence.
Suppose that we had independent data from an $$Exp(\theta_0)$$ distribution. In a Bayesian framework, we suppose that a priori $$\theta_0\sim Exp(\theta_1)$$, and that with uncertainty on the hyperparameter $$\theta_1$$, we might also give it a prior, $$\theta_1\sim Exp(\theta_2)$$ say. In fact at each level of the hierarchy we can hedge our bets by imposing a further level of prior uncertainty. Suppose we impose $$N$$ levels of the hierarchy by fixing the hyperparameter $$\theta_N$$ and sequentially setting $$\theta_i\sim Exp(\theta_{i+1})\qquad i=N-1,N-2,..., 1, 0$$ In terms of the data, the only thing that matters is the marginal prior of $$\theta_0$$, obtained (if it were possible) by integrating out the hierarchical parameters.In such a situation, it is natural to consider the prior distribution of $$\theta$$ as $$N\uparrow\infty$$. In this case and many others, no proper distributional limit exists, but the limit can sometimes still be described in terms of an improper prior distribution.