I am interested in finding an upper bound for

$\mathbb{E}\big( \int_B \frac{f(X| \theta)}{f(X| \theta_0)} \pi(\theta) d \theta\big)$,

the $\frac{f(X| \theta)}{f(X| \theta_0)}$ is the likelihood ratio. $\pi(\theta)$ is the prior for $\theta$ and $\theta \in B$.

Let $\Pi(\theta)$ be the CDF of $\pi(\theta)$.

Is it true the expectation is bounded by $\Pi(B)$? Why?

  • $\begingroup$ What is the CDF of a set $B$? So would that translate to $P(\theta\in B)$? $\endgroup$ – Alex R. Oct 17 '17 at 18:32
  • $\begingroup$ Yes, $\Pi(B)$ refers to $\Pi(\theta \in B)$. $\endgroup$ – Po Ning Oct 17 '17 at 18:36

The best possible bound is $c\Pi(B)$, where $c=\sup_{\theta,\theta_0}\frac{f(X|\theta)}{f(X|\theta_0)}$. As an example, suppose that $\theta$ is uniform in $[0,1]$, $B=[0,1]$, and $X$ is Bernoulli($\theta$). Then the likelihood ratio for say, $X=1$ would be $\theta/\theta_0$, so the integral would be:


which clearly depends on $\theta_0$. Now if you take the outside expectation, over $\theta_0$ the result can be anything. For example if $\theta_0$ follows $\pi(\theta_0)$ with the above uniform distribution, then the expectation would blowup.

  • $\begingroup$ you talk about $\mathbb{E}[U]$, but the OP talked about $\mathbb{E}\left[\int_B\frac{f(X| \theta)}{f(X| \theta_0)} \pi(\theta) d\theta\right]=\mathbb{E}[Y]$ where $Y=f(X)$ is the random variable $\left[\int_B\frac{f(X| \theta)}{f(X| \theta_0)} \pi(\theta) d\theta\right]$ (admittedly, a weird RV). I guess the OP notation is wrong. How did you interpret her/his question? $\endgroup$ – DeltaIV Oct 17 '17 at 18:47
  • $\begingroup$ The question is to find an upper bound for the expectation of the nominator of the posterior distribution $\pi(\theta \in B|X)$. Thus how the $\mathbb{E}\big( \int_B \frac{f(X|\theta)}{f(X|\theta_0)} \pi(\theta) d\theta \big)$ appears. $\endgroup$ – Po Ning Oct 17 '17 at 19:00
  • $\begingroup$ @DeltaIV: $E[\frac{f(X|\theta)}{f(X|\theta_0)}\pi(\theta)d\theta]=\frac{1}{\theta_0}E[U]$ in my setup, unless I made a mistake. $\endgroup$ – Alex R. Oct 17 '17 at 20:30
  • $\begingroup$ @AlexR. you missed an integral sign in your comment, but leaving that aside, I'm not sure how you prove that $\mathbb{E}\left[\int_B\frac{f(X| \theta)}{f(X| \theta_0)} \pi(\theta) d\theta\right]=\frac{1}{\theta_0}\mathbb{E}[U]$ in your context. Are you considering $X$ to be a random variable or a realization of a random variable, i.e., a constant?If $X$ is a constant, there's nothing random in $\int_B\frac{f(X| \theta)}{f(X| \theta_0)} \pi(\theta) d\theta$, so I don't understand what we are supposed to take the expectation of. If $X$ is random, then... $\endgroup$ – DeltaIV Oct 18 '17 at 9:27
  • 1
    $\begingroup$ @DeltaIV: I see what you mean, for some reason I thought the expectation outside was redundant. But then the question makes no sense because there's now a mystery distribution for $\theta_0$, unless $\theta_0$ also follows $\pi(\theta_0)$, in which case the conclusion is false since the above expectation would simply blowup. $\endgroup$ – Alex R. Oct 18 '17 at 17:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.