Proving that a function is always increasing

Question

Given that $X_1, \ldots, X_n$ are conditionally independent and identically distributed random variables and that, given a value of $\theta$, $X_i \mid \theta \sim \operatorname{Bernoulli}(\theta), i = 1, \ldots, n$. given that

$$r(i) = \mathbb{P}(X_i = 1 \mid X_1, \ldots, X_{i-1} = 1)$$

How do I prove that $r(i)$ is always increasing?

My approach

In order to prove that $r(i)$ is always increasing, I need to prove that $r(i) - r( i -1 )> 0, \forall i$. I know that $X \sim \operatorname{Bernoulli}(\theta)$, but i don't know what's the "closed form" of $r(i)$ for a given $i$.

Answer 1

By the law of iterative expectations and conditional independence, for any $n \in \mathbb{N}$ and $u_i \in \{0, 1\}$, $i = 1, \ldots, n$, we have \begin{align} & P(X_1 = u_1, \ldots X_n = u_n) = E[P(X_1 = u_1, \ldots, X_n = u_n|\theta)] \\ =& E[P(X_1 = u_1|\theta)\cdots P(X_n = u_n|\theta)] \\ =& E[\theta^s(1 - \theta)^{n - s}], \tag{1} \end{align} where $s = u_1 + \cdots + u_n$.

It then follows by $P(A|B) = P(A \cap B)/P(B)$ and $(1)$ that \begin{align} r(i) = \frac{P(X_1 = \cdots = X_i = 1)}{P(X_1 = \cdots = X_{i - 1} = 1)} = \frac{E[\theta^{i}]}{E[\theta^{i - 1}]}, \quad i = 1, 2, \ldots. \end{align}

So to show $r(i + 1) \geq r(i)$, it suffices to show that $(E[\theta^i])^2 \leq E[\theta^{i + 1}]E[\theta^{i - 1}]$. But this is a direct consequence of the Cauchy-Schwarz inequality: \begin{align} E[\theta^i] = E\left[\theta^{(i + 1)/2}\theta^{(i - 1)/2}\right] \leq \sqrt{E[\theta^{i + 1}]E[\theta^{i - 1}]}. \end{align}

This completes the proof.

Answer 2

It doesn't make sense to me that you say that they are independent, but then say that the conditional probability is increasing. If they are independent, then the conditional probability must be constant. What you seem to be getting at is that you're treating $\theta$ as being a random variable, but also a parameter, and knowing about $X_i$ for $i<n$ tells you something about $\theta$ and thus about $X_n$.

The probability that a variable with Bernoulli distribution of parameter $\theta$ will be $1$ is the same as the probability that a variable from a unit uniform distribution will be less than $\theta$, and if all of $X_1 ... X_{n-1}$ are less than $\theta$, then that means that $\max_{0<i<n}\{X_i\}<\theta$. If we label $\theta$ as $X_0$, then $\max_{0<i<n}\{X_i\}<\theta$ is the same as $\max_{0 \le i<n}\{X_i\} = X_0$. So now we have $r(n) = P(X_n < X_0 | \max_{0 \le i<n}\{X_i\} = X_0)$.

If we take $X_0$ to also be from the unit uniform distribution, I don't think we lose much generality. But it seems to me that there's now a symmetry argument to be made that that is the same as $r(n) = P(X_n < \max_{0 \le i<n}\{X_i\} )$. After all, why should $ P(X_n < X_0 | \max_{0 \le i<n}\{X_i\} = X_0)$ be any different from $ P(X_n < X_0 | \max_{0 \le i<n}\{X_i\} = X_k)$ for some $k$ where $0<k<n$? And if the probability is the same regardless of the condition, then we should be able to drop the condition.

But $P(X_n < \max_{0 \le i<n}\{X_i\} )$ is in turn the same as $P(\max_{0 \le i \le n}\{X_i\} \neq X_n)$. So now your question boils down to "Why is it that as we increase the number of variables, the probability that the last one is not the maximum is strictly increasing?" Or, alternatively, "Why is it that as we increase the number of variables, the probability that the last one is the maximum is decreasing?"

i don't know what's the "closed form" of r(i) for a given i

If you're treating $\theta$ as being a random variable, I think you need to know its distribution to calculate the closed form of $r(i)$. But the above argument shows that if it's a unit uniform distribution, then $r(i) = 1-\frac 1i$ or $\frac{i-1}i$