Bayesian updating with multiple priors and multiple likelihood functions

Question

$\Omega$ (finite) state space, $S$ (finite) signal space. Suppose we have a closed and convex set of priors $\mathcal{M}\subseteq \Delta(\Omega)$ such that each of them has full support. Let $\mathcal{L}\subseteq \{l:\Omega\to \Delta(S)\}$ be a closed and convex set of likelihood functions. Fix $s\in S$ and let $Pos(s)=\{\mu'\in \Delta(\Omega)|\mu'$ is the posterior generated by bayesian updating of some prior $\mu\in \mathcal{M}$, and $l\in \mathcal{L}$ conditional on $s\}$. My question is: is $Pos(s)$ convex?

Answer 1

I believe this is a counter-example showing that $Pos(s)$ is not necessarily convex.

Take $\Omega=\{1, 2, 3, 4\}$ and $\mathcal M$ the set of priors of the form

$$ \pi(1)=\pi(2)=\frac a2 \qquad \pi(3)=\pi(4)=\frac{1-a}{2}.$$

Let $\mathcal L$ be the set of likelihoods conditional on $s$ of the form $$l(1)=l(4)=b\qquad l(2)=l(3)=c.$$

Both $\mathcal M$ and $\mathcal L$ are convex. The posterior $\pi'$ is easy to compute: $$\pi'(1)=\frac{ab}{b+c}\qquad \pi'(2)=\frac{ac}{b+c}\qquad \pi'(3)=\frac{(1-a)c}{b+c}\qquad \pi'(4)=\frac{(1-a)b}{b+c}.$$

Note that $\forall \pi'\in Pos(s), \pi'(1)\pi'(3)=\pi'(2)\pi'(4)$.

In particular, if $a=\frac12, b=1, c=3$, $$\pi_0'(1)=\frac18\qquad \pi_0'(2)=\frac38\qquad \pi_0'(3)=\frac38\qquad \pi_0'(4)=\frac18$$

and if $a=\frac13, b=1, c=2$, $$\pi'_1(1)=\frac19\qquad\pi'_1(2)=\frac29\qquad\pi'_1(3)=\frac49\qquad\pi'_1(4)=\frac29.$$

Let $\pi'_\alpha=\frac{\pi'_0+\pi'_1}{2}$. Then $$\pi'_\alpha(1)=\frac{17}{144} \qquad\pi'_\alpha(2)=\frac{43}{144}\qquad\pi'_\alpha(3)=\frac{59}{144}\qquad\pi'_\alpha(4)=\frac{25}{144}.$$

Since $\pi'_\alpha(1)\pi'_\alpha(3)\neq \pi'_\alpha(2)\pi'_\alpha(4)$, we know that $\pi'_\alpha\not\in Pos(s)$, hence this set is not convex.

Answer 2

If $\pi_0,\pi_1\in\cal M$, then $\pi_\alpha=\alpha\pi_1+(1-\alpha)\pi_0\in\cal M$ for $0\le \alpha\le 1$. Considering the observation $s\in\cal S$ and $l\in\cal L$, denote $m_0(s),m_1(s)$ the associated marginal pdfs, $$m_0(s)=\mathbb E^{\pi_0}[l(|\theta|s)]\qquad m_1(s)=\mathbb E^{\pi_1}[l(|\theta|s)]$$The posterior associated with $\pi_\alpha$ is then $$\pi_\alpha(\cdot|s)=\frac{\alpha m_1(s)\pi_1(\cdot|s)+(1-\alpha) m_0(s)\pi_0(\cdot|s)}{\alpha m_1(s)+(1-\alpha)\pi_0(s)}$$a convex combination of the two posteriors. Conversely, considering the convex combination of the two posteriors$$\beta \pi_1(\cdot|s)+(1-\beta) \pi_0(\cdot|s)\qquad0\le\beta\le 1$$ there exists an $0\le \alpha\le 1$ such that $$\beta=\frac{\alpha m_1(s)}{\alpha m_1(s)+(1-\alpha)\pi_0(s)}$$because both $m_0(s)$ and $m_1(s)$ are different from zero by the full support assumption (assuming that $s$ is such that $l(s|\theta)>0$ for at least one value of $\theta\in\Omega$). The fact that the solution in $\alpha$ depends on $s$ is not an issue since the whole of $Pos(s)$ depends on $s$.

The same reasoning applies when only the likelihood functions, say $l_0$ versus $l_1$, in the two posteriors differ.

However, if both the likelihood functions and the priors in the two posteriors differ, it is unclear to me whether the convex combination $$\beta \pi_1(\cdot|s)+(1-\beta) \pi_0(\cdot|s)= \beta \frac{\pi_1(\cdot)l_1(s|\cdot)}{m_1(s)}+(1-\beta) \frac{\pi_0(\cdot)l_0(s|\cdot)}{m_0(s)}$$ can still be expressed as a convex combination of elements of $Pos(s)$. In the special case when both $\Omega$ and $S$ have only two elements, the result seems to hold.