Question about point mass prior and continuous distribution Suppose we have a point mass prior,
$$\theta \sim \begin{cases}  I(\theta=1)  ,& prob=\frac{1}{2} \\ 
Gamma(c,c), & prob=\frac{1}{2} \end{cases}$$
Then if we are asked 
$\lim_{c \to \infty} P(\theta=1)$
Now here is the issue, since Gamma is a continuous distribution, it seems that in the case of gamma,we will never have $\theta=1$.
To me it thus seems that regardless of the value of c, the $p(\theta=1)=\frac{1}{2}$
However, we also have that since expected value of a $gamma(a,b)=\frac{a}{b}$ so that the expected value of the gamma is 1 when we have $a=b=c$
But, by Markov, for $X \sim Gamma(c,c)$
$\lim_{c \to \infty} Pr(|X-\mu| \lt \epsilon) \to 1$ for any $\epsilon \gt 0$
So is $\lim_{c \to \infty}P(\theta=1) =1$ , or is  $\lim_{c \to \infty}P(\theta=1)=\frac{1}{2}$
As even though the markov inequality holds, it is a continous distirbution, so we will never have it exactly equal to 1.
Thanks all
 A: Your specified prior distribution is a mixture of a continuous and discrete part.  Let $I \sim \text{Bern}(1/2)$ be the indicator that the parameter is taken from the gamma distribution.  Then using the law of total probability you have:
$$\begin{equation} \begin{aligned}
\mathbb{P}(\theta = 1|c) 
&= \mathbb{P}(\theta = 1|c, I=0) \cdot \mathbb{P}(I=0) + \mathbb{P}(\theta = 1|c, I=1) \cdot \mathbb{P}(I=1) \\[6pt]
&= \frac{1}{2} \cdot \mathbb{P}(\theta = 1|c, I=0) + \frac{1}{2} \cdot \mathbb{P}(\theta = 1|c, I=1) \\[6pt]
&= \frac{1}{2} + \frac{1}{2} \cdot \mathbb{P}(\theta = 1| \theta \sim \text{Ga}(c,c)) \\[6pt]
&= \frac{1}{2}. \\[6pt]
\end{aligned} \end{equation}$$
(Note that the last step comes from recognising that the gamma is a continuous distribution, so the probability of a specific point is zero under this distribution.)  This result holds for all values of $c$, so you are correct that it also holds in the limit:
$$\lim_{c \rightarrow \infty} \mathbb{P}(\theta = 1|c) = \lim_{c \rightarrow \infty} \frac{1}{2} = \frac{1}{2}.$$
Your later use of Chebychev's inequality shows that as $c \rightarrow \infty$ you get $\theta \rightarrow 1$ (convergence in probability), but this does not change the fact that $\mathbb{P}(\theta = 1|c) = 1/2$ for all $c > 0$.  (To understand the reason for this more fully, have a read about the distinction between convergence in probability and almost-sure convergence.  Presumably the example is selected precisely to illustrate this distinction.)
A: 
Disclaimer The above answer is completely fine if the question
is about the prior probability that $θ=1$. While there is no
indication whatsoever in the text of the question to the alternative, namely
that the question is about the posterior probability that $θ=1$, a recent
exchange with students about a very similar question in my book, The
Bayesian Choice (Exercise 5.9), leads me to propose the extension
to the posterior setting.

In this posterior case, when $X\sim \mathcal{E}(\theta)$ [for instance] the probability that $θ=1$ is
\begin{align*}
\mathbb{P}(\theta=1|x)&= \frac{e^{-x}}{e^{-x}+\int_0^\infty \theta e^{-\theta x} \pi_c(\theta)\text{d}\theta}\\
&=\frac{e^{-x}}{e^{-x}+ \int_0^\infty \theta e^{-\theta x} c^c \theta^{c-1}e^{-c \theta} \Gamma(c)^{-1}\text{d}\theta}\\
&=\frac{e^{-x}}{e^{-x}+c^c (c+x)^{-c-1} \Gamma(c+1)\Gamma(c)^{-1}}\\
&=\frac{e^{-x}}{e^{-x}+c^{c+1} (c+x)^{-c-1}}\\
&=\frac{e^{-x}}{e^{-x}+ (1+x/c)^{-c-1}}\\
\end{align*}
which converges to
$$\frac{e^{-x}}{e^{-x}+e^{-x}}=\frac{1}{2}$$
when $c$ goes to infinity.
