I am reading this article:

Where as far as I can make out they are taking the expected value of the difference of 2 conditional probabilities (p. 158):


Where they have defined $P(C=c|Y=1)$ as well as $P(C=c|Y=0)$. So I guess we can use the fact that the expected is a linear operator and get:


But I still do not really understand how we could compute $E[P(C=c|Y=1)]$, as it is just a single value, would it just be:


Or I guess perhaps a broader point (in case you do not understand this example). Is how to take the expected value of probabilities, as usually you use the probabilities as weights, when you are calculating the mean. Or at least how to think of expected values in this case?


So I just realised another expected value that I do not quite understand (p. 159-160), they have logistic model:


Where we should note that the distribution of $Y|X_1,...,X_L$ equals the distribution of $Y|X_1,...,X_L,C$. They then they claim that the expected value is:


How come the $X$s do not affect the mean value? Or you just show me how they derive this?

  • 1
    $\begingroup$ Are you sure you typed that first formula in correctly? As it is written, it equals 0, as $\sum_cP(C=c|Y=1) = 1$ (after all, $C$ has to equal something), and similarly for the other term... also, not everyone has access to that paper (me for example), so you might want to excerpt a bit more, if the math is relevant. $\endgroup$
    – jbowman
    Commented Sep 16, 2018 at 17:29

1 Answer 1


I'll preface my answer by saying that I have not thought about the term "allele" since high school, so please let me know if I am somehow misrepresenting what the paper said when I quickly skimmed the page you mentioned. Anyway:

I believe where you are confused is when they claim that $E[\delta]=0$, where $$\delta = \sum_c p_cd_c= \sum_cP(G=A\mid C=c)[P(C=c\mid Y=1)-P(C=c\mid Y=0)]$$

Importantly, the authors assume that "$p_c=P(A\mid C=c)$ is an i.i.d. random variable, c=1, ..., m, with mean p and variance $F_{st}p(1 − p)$." So now $p_c$ is a random variable that follows some distribution! On the other hand, $d_c$ is not your typical random variable, though it technically can be thought as one where it will always be the same value (see https://en.wikipedia.org/wiki/Degenerate_distribution). With this in mind, the problem is simple:

First note, as @jbowman mentions in his comment, that $$\sum_cd_c =\sum_c(P(C=c\mid Y=1)) - \sum_c(P(C=c\mid Y=0)) = 1-1=0$$ because you are summing over all $C$ (this is a basic axiom of probability). Note that $\sum_cp_cd_c \neq 0$. Then: $$\begin{align} E[\delta]= & E[\sum_cp_cd_c]\\ = & \sum_cE[p_cd_c] \\ = & \sum_cE[p_c]d_c \text{ (because $d_c$ is a constant)}\\ = & \sum_cpd_c = p\sum_c d_c = 0 \end{align}$$ because we could take out the $p_c$s.

So I think your intuition is mostly correct... a probability can easily be a random variable, but often it is a constant value, which means that it's expected value is equal to the probability itself.

When thinking about expected value of probability as in the above, imagine actually calculating each $P(A\mid C=c) \forall c \in \{1,...n\}$. It's very conceivable that if you sampled many many such probabilities, they'd form some distribution, and naturally will have a mean and variance. I believe the reason they have this probability follow a distribution in your paper is because C is ultimately unobservable, but I could be wrong...

Regarding your edit, recall that Y is a binary variable (i totally forgot this until skimming the paper again.. its important!). Because of this, note that $E[Y\mid X] = P(Y=1\mid X)$ Why? Because Y is either 1 or 0, the expected value of Y is also the probability of Y=1. With this in mind, and recalling the Law of Iterated Expectation (LIE, which says $E[E[Y\mid X]] = E[Y]$), let's work this through (I'll work backwards in your problem, where for simplicity I consider a single X): $$\begin{align} E[P(Y=1\mid X)] & = E[P(Y=1\mid X,C=c)] \\ & = E[E[Y=1\mid X,C=c]] \\ & = E[Y=1\mid C=c] \text{ by LIE!!} \\ & = P(Y=1\mid C=c] \text{ again abusing binaryness of Y} \end{align}$$

Ta-Daa!!! This took me a while to think through, because I forgot that Y is a binary variable. This cannot be done if it was not.

  • 1
    $\begingroup$ Hi thanks you so much for your very nice answer, this now makes sense to me! :) I have added a second question to my original post, I hope this is OK conduct on here. $\endgroup$
    – lo2
    Commented Sep 19, 2018 at 21:09
  • $\begingroup$ I updated my answer for your question. Generally it should have been a new post because the intuition behind it was different. Please accept an answer when you are satisfied so the community knows you got your answer, and you can thumbs up or down it depending on how you liked it. Hope this helps :)! $\endgroup$
    – doubled
    Commented Sep 19, 2018 at 22:25
  • $\begingroup$ Sure, I have accepted your answer! Thank you so much :) You are awesome $\endgroup$
    – lo2
    Commented Sep 20, 2018 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.