Expected value of a conditional probability

Question

I am reading this article:
https://www.sciencedirect.com/science/article/pii/S0040580901915424?via%3Dihub

Where as far as I can make out they are taking the expected value of the difference of 2 conditional probabilities (p. 158):

$E[\sum_c{P(C=c|Y=1)-P(C=c|Y=0)}]$

Where they have defined $P(C=c|Y=1)$ as well as $P(C=c|Y=0)$. So I guess we can use the fact that the expected is a linear operator and get:

$\sum_c{E[P(C=c|Y=1)]}-\sum_c{E[P(C=c|Y=0)]}$

But I still do not really understand how we could compute $E[P(C=c|Y=1)]$, as it is just a single value, would it just be:

$E[P(C=c|Y=1)]=P(C=c|Y=1)$

Or I guess perhaps a broader point (in case you do not understand this example). Is how to take the expected value of probabilities, as usually you use the probabilities as weights, when you are calculating the mean. Or at least how to think of expected values in this case?

#### EDIT: ADDED ANOTHER QUESTION

So I just realised another expected value that I do not quite understand (p. 159-160), they have logistic model:

$P(Y=1|X_1,...,X_L)=\frac{e^{\beta_0+\sum_{l=1}^L{\beta_lX_{il}}}}{1+e^{\beta_0+\sum_{l=1}^L{\beta_lX_{il}}}}$

Where we should note that the distribution of $Y|X_1,...,X_L$ equals the distribution of $Y|X_1,...,X_L,C$. They then they claim that the expected value is:

$P(Y=1|C=c)=r_c=E[P(Y=1|X_1,...,X_L)]=E[\frac{e^{\beta_0+\sum_{l=1}^L{\beta_lX_{il}}}}{1+e^{\beta_0+\sum_{l=1}^L{\beta_lX_{il}}}}]$

How come the $X$s do not affect the mean value? Or you just show me how they derive this?

Answer 1

I'll preface my answer by saying that I have not thought about the term "allele" since high school, so please let me know if I am somehow misrepresenting what the paper said when I quickly skimmed the page you mentioned. Anyway:

I believe where you are confused is when they claim that $E[\delta]=0$, where $$\delta = \sum_c p_cd_c= \sum_cP(G=A\mid C=c)[P(C=c\mid Y=1)-P(C=c\mid Y=0)]$$

Importantly, the authors assume that "$p_c=P(A\mid C=c)$ is an i.i.d. random variable, c=1, ..., m, with mean p and variance $F_{st}p(1 − p)$." So now $p_c$ is a random variable that follows some distribution! On the other hand, $d_c$ is not your typical random variable, though it technically can be thought as one where it will always be the same value (see https://en.wikipedia.org/wiki/Degenerate_distribution). With this in mind, the problem is simple:

First note, as @jbowman mentions in his comment, that $$\sum_cd_c =\sum_c(P(C=c\mid Y=1)) - \sum_c(P(C=c\mid Y=0)) = 1-1=0$$ because you are summing over all $C$ (this is a basic axiom of probability). Note that $\sum_cp_cd_c \neq 0$. Then: $$\begin{align} E[\delta]= & E[\sum_cp_cd_c]\\ = & \sum_cE[p_cd_c] \\ = & \sum_cE[p_c]d_c \text{ (because $d_c$ is a constant)}\\ = & \sum_cpd_c = p\sum_c d_c = 0 \end{align}$$ because we could take out the $p_c$s.

So I think your intuition is mostly correct... a probability can easily be a random variable, but often it is a constant value, which means that it's expected value is equal to the probability itself.

When thinking about expected value of probability as in the above, imagine actually calculating each $P(A\mid C=c) \forall c \in \{1,...n\}$. It's very conceivable that if you sampled many many such probabilities, they'd form some distribution, and naturally will have a mean and variance. I believe the reason they have this probability follow a distribution in your paper is because C is ultimately unobservable, but I could be wrong...

Regarding your edit, recall that Y is a binary variable (i totally forgot this until skimming the paper again.. its important!). Because of this, note that $E[Y\mid X] = P(Y=1\mid X)$ Why? Because Y is either 1 or 0, the expected value of Y is also the probability of Y=1. With this in mind, and recalling the Law of Iterated Expectation (LIE, which says $E[E[Y\mid X]] = E[Y]$), let's work this through (I'll work backwards in your problem, where for simplicity I consider a single X): $$\begin{align} E[P(Y=1\mid X)] & = E[P(Y=1\mid X,C=c)] \\ & = E[E[Y=1\mid X,C=c]] \\ & = E[Y=1\mid C=c] \text{ by LIE!!} \\ & = P(Y=1\mid C=c] \text{ again abusing binaryness of Y} \end{align}$$

Ta-Daa!!! This took me a while to think through, because I forgot that Y is a binary variable. This cannot be done if it was not.