1
$\begingroup$

Suppose we have a sequence of tuples of random variables $(X_i,Y_i)$ where $X_i \sim \text{Bern}(p_i)$ and $Y_i \sim \text{Bern}(q_i)$ are correlated, where $p_i$ is unknown but $q_i$ is known. We get to observe $n$ of these tuples $(x_i,y_i)$. The problem is to optimally estimate $p_i$ as a function of $q_i$. Any references on the problem would be appreciated, as I don't even know which field this belongs to, let alone if the specific problem has a name.

Example: Given the probability that a woman gives birth to four or more children in her lifetime ($Y_i$ is the indicator of this event), we want to estimate the probability that she gives birth at all before age 20 ($X_i$ is indicator), where $q_i$ is allowed to be some arbitrary function of income, education level, geographic location or whatever. The main point is that it is assumed to be known.

EDIT: Maybe it needs to be stated explicitly that $p_i \neq p_j, q_i \neq q_j$ for $i \neq j$ is entirely possible. We don't have a sequence of observations from the same distribution, they're all from (potentially) different distributions. Given $n$ observations of $(X_i,Y_i)$, we want to estimate $P(X_{n+1} = 1)$ given perfect knowledge of the DISTRIBUTION of $Y_{n+1}$, but prior to actually observing it.

$\endgroup$
8
  • $\begingroup$ you have to specify how X and Y are 'correlated' otherwise one cannot optimally estimate. one example might simply be that X is simply logistic function of Y... $\endgroup$ – seanv507 Aug 17 '16 at 20:30
  • $\begingroup$ Are you saying there is no optimal solution to this estimation problem given no prior knowledge on the exact correlation...? That sounds strong. $\endgroup$ – Benjamin Lindqvist Aug 17 '16 at 20:32
  • 1
    $\begingroup$ Is this your situation? You know P(Y). You have observations of (X,Y). And you want to know P(X|Y). If so, P(X|Y) = P(X,Y)//P(Y). Estimate P(X,Y) from the data.for Y = y values of 0 and 1, and calculate the corresponding P(X|Y). Or something along those lines. But I',m not sure I'm understanding your actual situation. $\endgroup$ – Mark L. Stone Aug 17 '16 at 20:36
  • $\begingroup$ Yeah sort of but $X$ and $Y$ aren't uniform iid - we know $P(Y_i)$ and want to estimate $P(X_i|Y_i)$ given $P(Y_i)$ based on $n$ observations of $(X_i,Y_i)$ that are possibly all from different distributions. That's the catch. $\endgroup$ – Benjamin Lindqvist Aug 17 '16 at 20:44
  • 1
    $\begingroup$ You have (X,Y) as ordered pairs. That is (should be) taken into account when estimating P(X,Y) from the data. The marginal distributions of X and Y, whatever they happen to be, even if different, and whatever their dependency, are"automatically" modeled by "nature" and reflected in the paired data. $\endgroup$ – Mark L. Stone Aug 17 '16 at 20:50
1
$\begingroup$

My interpretation of your situation: $X$ and $Y$ each take the values 0 or 1. You know $P(Y)$,i.e., $P(Y=1) = q$. You have observations of $(X,Y)$. And you want to know $P(X|Y)$. If so, make use of $$P(X|Y) = \frac{P(X,Y)}{P(Y)}$$Estimate P(X,Y) from the data.for Y = y values of 0 and 1, and calculate the corresponding $P(X|Y)$.

Note that you have $(X,Y)$ as ordered pairs. That is (should be) taken into account when estimating $P(X,Y)$ from the data. The marginal distributions of $X$ and $Y$, whatever they happen to be, even if different, and whatever their dependency, are"automatically" modeled by "nature" and reflected in the paired data.

Explicit Solution:

$P(X=1|Y=1)$ is estimated as $$\frac{P(X=1,Y=1)}{q}$$ where $P(X=1,Y=1)$ is the sample fraction of all pairs in which both $X=1$ and $Y=1$.

$P(X=1|Y=0)$ is estimated as $$\frac{P(X=1,Y=0)}{(1-q)}$$ where $P(X=1,Y=0)$ is the sample fraction of all pairs in which both $X=1$ and $Y=0$.

$P(X=0|Y=1) = 1-P(X=1|Y=1)$ and $P(X=0|Y=0) = 1-P(X=1|Y=0)$ where the estimates on the left-hand sides are obtained using the estimates on the right-hand side.

Addressing a comment: We also have $P(X=1) = P(X=1|Y=1)q + P(X=1|Y=0)(1-q)$, so we can estimate the left-hand side using estimated values on the right-hand side Note however, that we could have estimated just $P(X=1)$ more directly as the sample fraction of all pairs $(X,Y)$ having $X=1$. And of course $P(X=0) = 1-P(X=1)$.

$\endgroup$
12
  • $\begingroup$ I'm actually not sure anymore if I'm not getting something or if there's a miscommunication on my part. To be clear, I want to estimate $P(X_i)$ but I can't actually condition on $Y_i$ - I can only condition on $X_1,\dots,X_{i-1}, Y_1,\dots, Y_{i-1}$ as well as the true distribution of $Y_1,\dots,Y_{i-1}$. Are we on the same page here? $\endgroup$ – Benjamin Lindqvist Aug 17 '16 at 22:00
  • $\begingroup$ Like, estimating $P(X,Y)$ as the fraction of observations such that $X=1,Y=1$ is kind of dubious, because I only have one observation of each tuple since they're not iid. $\endgroup$ – Benjamin Lindqvist Aug 17 '16 at 22:03
  • $\begingroup$ Are the pairs $(X_i,Y_i)$ iid across different values of $i$, even though $X_i$ and $Y_i$ are not independent for each given value of $i$? When I referred to observations such that $X=1,Y=1$, I meant by that, all the values of $i$ such that $X_i=1,Y_i=1$ and assumed that $i$ is not some tiny number, such as 1 or 4. $\endgroup$ – Mark L. Stone Aug 17 '16 at 22:26
  • $\begingroup$ $X_i$ and $Y_j$ are independent for $i \neq j$, yes. But estimating $P(X_i,Y_i)$ as the fraction of $(1,1)$ observations in total does not reflect that they're not identically distributed... $\endgroup$ – Benjamin Lindqvist Aug 17 '16 at 22:32
  • $\begingroup$ I still don't understand what you're saying. Consider (X,Y) to be a bivariate random variable. Do you have n observations of this bivariate random variable, $(X_i,Y_i)$ for $i$ from $1$ to $n$, and $(X_i,Y_i)$ are independent across different values of $i$? Note that this still allows $X$ and $Y$ to be dependent and to have different marginal distributions. $\endgroup$ – Mark L. Stone Aug 17 '16 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.