# Estimating probabilities given the probabily of correlated event

Suppose we have a sequence of tuples of random variables $(X_i,Y_i)$ where $X_i \sim \text{Bern}(p_i)$ and $Y_i \sim \text{Bern}(q_i)$ are correlated, where $p_i$ is unknown but $q_i$ is known. We get to observe $n$ of these tuples $(x_i,y_i)$. The problem is to optimally estimate $p_i$ as a function of $q_i$. Any references on the problem would be appreciated, as I don't even know which field this belongs to, let alone if the specific problem has a name.

Example: Given the probability that a woman gives birth to four or more children in her lifetime ($Y_i$ is the indicator of this event), we want to estimate the probability that she gives birth at all before age 20 ($X_i$ is indicator), where $q_i$ is allowed to be some arbitrary function of income, education level, geographic location or whatever. The main point is that it is assumed to be known.

EDIT: Maybe it needs to be stated explicitly that $p_i \neq p_j, q_i \neq q_j$ for $i \neq j$ is entirely possible. We don't have a sequence of observations from the same distribution, they're all from (potentially) different distributions. Given $n$ observations of $(X_i,Y_i)$, we want to estimate $P(X_{n+1} = 1)$ given perfect knowledge of the DISTRIBUTION of $Y_{n+1}$, but prior to actually observing it.

• you have to specify how X and Y are 'correlated' otherwise one cannot optimally estimate. one example might simply be that X is simply logistic function of Y... – seanv507 Aug 17 '16 at 20:30
• Are you saying there is no optimal solution to this estimation problem given no prior knowledge on the exact correlation...? That sounds strong. – Benjamin Lindqvist Aug 17 '16 at 20:32
• Is this your situation? You know P(Y). You have observations of (X,Y). And you want to know P(X|Y). If so, P(X|Y) = P(X,Y)//P(Y). Estimate P(X,Y) from the data.for Y = y values of 0 and 1, and calculate the corresponding P(X|Y). Or something along those lines. But I',m not sure I'm understanding your actual situation. – Mark L. Stone Aug 17 '16 at 20:36
• Yeah sort of but $X$ and $Y$ aren't uniform iid - we know $P(Y_i)$ and want to estimate $P(X_i|Y_i)$ given $P(Y_i)$ based on $n$ observations of $(X_i,Y_i)$ that are possibly all from different distributions. That's the catch. – Benjamin Lindqvist Aug 17 '16 at 20:44
• You have (X,Y) as ordered pairs. That is (should be) taken into account when estimating P(X,Y) from the data. The marginal distributions of X and Y, whatever they happen to be, even if different, and whatever their dependency, are"automatically" modeled by "nature" and reflected in the paired data. – Mark L. Stone Aug 17 '16 at 20:50

My interpretation of your situation: $X$ and $Y$ each take the values 0 or 1. You know $P(Y)$,i.e., $P(Y=1) = q$. You have observations of $(X,Y)$. And you want to know $P(X|Y)$. If so, make use of $$P(X|Y) = \frac{P(X,Y)}{P(Y)}$$Estimate P(X,Y) from the data.for Y = y values of 0 and 1, and calculate the corresponding $P(X|Y)$.

Note that you have $(X,Y)$ as ordered pairs. That is (should be) taken into account when estimating $P(X,Y)$ from the data. The marginal distributions of $X$ and $Y$, whatever they happen to be, even if different, and whatever their dependency, are"automatically" modeled by "nature" and reflected in the paired data.

Explicit Solution:

$P(X=1|Y=1)$ is estimated as $$\frac{P(X=1,Y=1)}{q}$$ where $P(X=1,Y=1)$ is the sample fraction of all pairs in which both $X=1$ and $Y=1$.

$P(X=1|Y=0)$ is estimated as $$\frac{P(X=1,Y=0)}{(1-q)}$$ where $P(X=1,Y=0)$ is the sample fraction of all pairs in which both $X=1$ and $Y=0$.

$P(X=0|Y=1) = 1-P(X=1|Y=1)$ and $P(X=0|Y=0) = 1-P(X=1|Y=0)$ where the estimates on the left-hand sides are obtained using the estimates on the right-hand side.

Addressing a comment: We also have $P(X=1) = P(X=1|Y=1)q + P(X=1|Y=0)(1-q)$, so we can estimate the left-hand side using estimated values on the right-hand side Note however, that we could have estimated just $P(X=1)$ more directly as the sample fraction of all pairs $(X,Y)$ having $X=1$. And of course $P(X=0) = 1-P(X=1)$.

• I'm actually not sure anymore if I'm not getting something or if there's a miscommunication on my part. To be clear, I want to estimate $P(X_i)$ but I can't actually condition on $Y_i$ - I can only condition on $X_1,\dots,X_{i-1}, Y_1,\dots, Y_{i-1}$ as well as the true distribution of $Y_1,\dots,Y_{i-1}$. Are we on the same page here? – Benjamin Lindqvist Aug 17 '16 at 22:00
• Like, estimating $P(X,Y)$ as the fraction of observations such that $X=1,Y=1$ is kind of dubious, because I only have one observation of each tuple since they're not iid. – Benjamin Lindqvist Aug 17 '16 at 22:03
• Are the pairs $(X_i,Y_i)$ iid across different values of $i$, even though $X_i$ and $Y_i$ are not independent for each given value of $i$? When I referred to observations such that $X=1,Y=1$, I meant by that, all the values of $i$ such that $X_i=1,Y_i=1$ and assumed that $i$ is not some tiny number, such as 1 or 4. – Mark L. Stone Aug 17 '16 at 22:26
• $X_i$ and $Y_j$ are independent for $i \neq j$, yes. But estimating $P(X_i,Y_i)$ as the fraction of $(1,1)$ observations in total does not reflect that they're not identically distributed... – Benjamin Lindqvist Aug 17 '16 at 22:32
• I still don't understand what you're saying. Consider (X,Y) to be a bivariate random variable. Do you have n observations of this bivariate random variable, $(X_i,Y_i)$ for $i$ from $1$ to $n$, and $(X_i,Y_i)$ are independent across different values of $i$? Note that this still allows $X$ and $Y$ to be dependent and to have different marginal distributions. – Mark L. Stone Aug 17 '16 at 22:38