4
$\begingroup$

Consider the following model:

$$y \sim \text{Exponential}(\lambda_0) \\ \lambda_i | \lambda_{i+1} \sim \text{Exponential}(\lambda_i+1) \\ \text{for } i=1,2,\dots,d\\ \lambda_{d+1} = k $$ With an observed $y=c$.

How does the marginal likelihood given a fixed number $d$ of priors $P_d(y)$ behave as a function of $d$? How about $\lim_{d\to\infty} P_d(y)$

Can a general statement be made for arbitrary distributions?

$\endgroup$
2
  • 1
    $\begingroup$ projecteuclid.org/euclid.bj/1080004760 investigates these sorts of questions $\endgroup$
    – Cyan
    Oct 22 '20 at 13:13
  • 1
    $\begingroup$ @Cyan oh wow there's actually a paper on this exact question. I read the abstract, this is definitely going on my steadily growing reading list! Thanks so much for sharing! $\endgroup$ Oct 22 '20 at 22:01
2
$\begingroup$

The parameter of the Exponential distribution is an inverse scale parameter: $X\sim \mathcal E(\lambda)$ can be represented as$$X=\epsilon/\lambda\qquad\epsilon\sim\mathcal E(1)$$Therefore if $$X \sim \mathcal E(\lambda_0) \\ \lambda_i | \lambda_{i+1} \sim \mathcal E(\lambda_{i+1}) \quad i=0,1,\dots,d\\ \lambda_{d+1} = 1$$ we can write \begin{align}X &= \epsilon_0 / \lambda_0 \qquad &\epsilon_0\sim\mathcal E(1)\\ &= \lambda_1 \epsilon_0 / \epsilon_1 \qquad &\epsilon_1\sim\mathcal E(1)\\ &= \epsilon_2 \epsilon_0 / \epsilon_1 \lambda_2 \qquad &\epsilon_2\sim\mathcal E(1)\\ &\quad \vdots\\ &= \frac{\epsilon_{2\lfloor d/2 \rfloor}\cdots \epsilon_0}{\epsilon_{2\lfloor (d-1)/2 \rfloor+1}\cdots \epsilon_1}\qquad &\epsilon_d\sim\mathcal E(1) \end{align} and hence \begin{align*}X&=\frac{\exp\left\{\sum_{i=0}^{\lfloor d/2 \rfloor}\log\epsilon_{2i}\right\}}{\exp\left\{\sum_{i=0}^{\lfloor (d-1)/2 \rfloor}\log\epsilon_{2i+1}\right\}}\qquad \epsilon_i\sim\mathcal E(1)\\ &=\frac{\exp\left\{\sum_{i=0}^{\lfloor d/2 \rfloor}\log\{-\log\upsilon_{2i}\}\right\}}{\exp\left\{\sum_{i=0}^{\lfloor (d-1)/2 \rfloor}\log\{-\log\upsilon_{2i+1}\}\right\}} \qquad \upsilon_i\sim\mathcal U(0,1)\end{align*} appears as the ratio of two independent random variables both converging a.s. to zero with $d$, hence having no limiting distribution. (This property generalises to other scale distributions, obviously, provided $\mathbb E[\sqrt{\epsilon_i}]<1$.)

Considering the general case $\lambda_d\sim\mathcal E(k)$ simply means alternatively multiplying and dividing by $k$ the above, hence preventing any form of convergence.

Note [update]: Corey Yanovski pointed out to me that this problem is investigated in “Infinite hierarchies and prior distributions” by Gareth O. Roberts and Jeffrey S. Rosenthal (Bernoulli, 2001). The introduction reads as follows:

Suppose that we had independent data from an $Exp(\theta_0)$ distribution. In a Bayesian framework, we suppose that a priori $\theta_0\sim Exp(\theta_1)$, and that with uncertainty on the hyperparameter $\theta_1$, we might also give it a prior, $\theta_1\sim Exp(\theta_2)$ say. In fact at each level of the hierarchy we can hedge our bets by imposing a further level of prior uncertainty. Suppose we impose $N$ levels of the hierarchy by fixing the hyperparameter $\theta_N$ and sequentially setting $$\theta_i\sim Exp(\theta_{i+1})\qquad i=N-1,N-2,..., 1, 0$$ In terms of the data, the only thing that matters is the marginal prior of $\theta_0$, obtained (if it were possible) by integrating out the hierarchical parameters.In such a situation, it is natural to consider the prior distribution of $\theta$ as $N\uparrow\infty$. In this case and many others, no proper distributional limit exists, but the limit can sometimes still be described in terms of an improper prior distribution.

$\endgroup$
1
  • $\begingroup$ are there any general statements or results relating to larger and larger hierarchical models? thanks for the answer, that was a clever way to go about it... $\endgroup$ Oct 10 '20 at 2:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.