Behaviour of the marginal in the limit for an infinite sequence of hierarchical priors Consider the following model:
$$y \sim \text{Exponential}(\lambda_0) \\
\lambda_i | \lambda_{i+1} \sim \text{Exponential}(\lambda_i+1) \\
\text{for } i=1,2,\dots,d\\
\lambda_{d+1} = k
$$
With an observed $y=c$.
How does the marginal likelihood given a fixed number $d$ of priors $P_d(y)$ behave as a function of $d$? How about $\lim_{d\to\infty} P_d(y)$
Can a general statement be made for arbitrary distributions?
 A: The parameter of the Exponential distribution is an inverse scale parameter: $X\sim \mathcal E(\lambda)$ can be represented as$$X=\epsilon/\lambda\qquad\epsilon\sim\mathcal E(1)$$Therefore if
$$X \sim \mathcal E(\lambda_0) \\
\lambda_i | \lambda_{i+1} \sim \mathcal E(\lambda_{i+1}) \quad
i=0,1,\dots,d\\
\lambda_{d+1} = 1$$
we can write
\begin{align}X &= \epsilon_0 / \lambda_0 \qquad &\epsilon_0\sim\mathcal E(1)\\
&= \lambda_1 \epsilon_0 / \epsilon_1 \qquad &\epsilon_1\sim\mathcal E(1)\\
&= \epsilon_2 \epsilon_0 / \epsilon_1 \lambda_2 \qquad &\epsilon_2\sim\mathcal E(1)\\
&\quad \vdots\\
&= \frac{\epsilon_{2\lfloor d/2 \rfloor}\cdots \epsilon_0}{\epsilon_{2\lfloor (d-1)/2 \rfloor+1}\cdots \epsilon_1}\qquad &\epsilon_d\sim\mathcal E(1)
\end{align}
and hence
\begin{align*}X&=\frac{\exp\left\{\sum_{i=0}^{\lfloor d/2 \rfloor}\log\epsilon_{2i}\right\}}{\exp\left\{\sum_{i=0}^{\lfloor (d-1)/2 \rfloor}\log\epsilon_{2i+1}\right\}}\qquad \epsilon_i\sim\mathcal E(1)\\
&=\frac{\exp\left\{\sum_{i=0}^{\lfloor d/2 \rfloor}\log\{-\log\upsilon_{2i}\}\right\}}{\exp\left\{\sum_{i=0}^{\lfloor (d-1)/2 \rfloor}\log\{-\log\upsilon_{2i+1}\}\right\}}
\qquad \upsilon_i\sim\mathcal U(0,1)\end{align*}
appears as the ratio of two independent random variables both converging a.s. to zero with $d$, hence having no limiting distribution. (This property generalises to other scale distributions, obviously, provided $\mathbb E[\sqrt{\epsilon_i}]<1$.)
Considering the general case $\lambda_d\sim\mathcal E(k)$ simply means alternatively multiplying and dividing by $k$ the above, hence preventing any form of convergence.
Note [update]: Corey Yanovski pointed out to me that this problem is investigated in “Infinite hierarchies and prior distributions” by Gareth O. Roberts and Jeffrey S. Rosenthal (Bernoulli, 2001). The introduction reads as follows:

Suppose   that   we   had   independent   data   from   an $Exp(\theta_0)$   distribution.   In   a   Bayesian framework,   we  suppose  that   a  priori $\theta_0\sim Exp(\theta_1)$,  and  that  with  uncertainty  on  the hyperparameter $\theta_1$,  we  might  also  give  it  a  prior, $\theta_1\sim Exp(\theta_2)$  say.  In  fact  at  each  level  of  the hierarchy we can hedge our bets by imposing a further level of prior uncertainty. Suppose we impose $N$ levels  of  the  hierarchy  by  fixing  the  hyperparameter $\theta_N$ and  sequentially  setting
$$\theta_i\sim Exp(\theta_{i+1})\qquad i=N-1,N-2,..., 1, 0$$ In terms of the data, the only thing that matters is the marginal prior of $\theta_0$,  obtained  (if  it  were  possible)  by  integrating  out  the  hierarchical  parameters.In  such  a  situation,  it  is  natural  to  consider  the  prior  distribution  of $\theta$ as $N\uparrow\infty$. In this  case  and  many others,  no  proper  distributional  limit  exists,  but  the  limit  can  sometimes still  be  described  in  terms  of  an  improper  prior  distribution.

