Creating conditional probability distribution

Question

I have a system which visits two locations and collects signals every 10 seconds. It is well known that A causes B to occur but not all the time. So, I am trying to find a conditional probability distribution of B given A.

The data looks like following:

A    B    
----------
1    0    
1    1    
1    0    
1    0    
..........

there are about 10,000 rows

How do I use the above to create a conditional distribution: P(B|A) i.e. probability of B hapenning given A.

@Coronoe: Yes I mean the uncertainty in Arthurs calculation, how do I achieve that. Do we fit a 4 case binomial distribution here? The 4 cases are mentioned by Arthur.

How does this change is I have instead:

A    B    C
------------
1    0    1
1    1    0
1    0    1
1    0    1
............

Now I want to understand P(B,C|A). @Corone I really like the way you have described the math and it just intrigued my interest as to what happens now? Again I am pretty much of novice in this field.

So, my understanding now is that I would have 8 cases here like the following: P(B=1,C=1|A=1), P(B=1,C=1|A=0), P(B=1,C=0|A=1), etc and I modify the equation of frequentist approach to consider above 8 cases? But does that take care of any correlation between B and C as well i.e how B effects the happening of C in addition to A? Or should I start to think in terms of Dirichlet distribution here since it is a joint probability distribution.

Answer 1

Picking up where Arther left off, assuming independence between samples, the completely saturated (most general) model for this type of data is to assign a different probability to each possible outcome $p_{00}, p_{10}, p_{01}$ and $p_{11}$ as defined by Arthur. Notice that the only restriction on what these values may take is

$p_{00}+p_{01}+p_{10}+p_{11}=1$

These probability could be estimated from your data simply by counting up the frequencies. Because of the constraint imposed, and the nature of the problem, the set of 4 probability estimates would be drawn from a Dirichlet distribution.

Now, in your case you are only interested in the two conditional probabilities $p_{1|1} = P(B|A=1)$ and $p_{1|0}=P(B|A=0)$. We can write down the likelihood function for these, given your data, as follows:

$P(p_{1|1} | \{n_{ij}\}) \propto p_{1|1}^{n_{11}} (1-p_{1|1})^{n_{10}}$

If we differentiate this with respect to $p_{1|1}$ then we find

$\frac{\partial L}{\partial p_{1|1}} = n_{11} p_{1|1}^{n_{11}-1} (1-p_{1|1})^{n_{10}} - n_{10} p_{1|1}^{n_{11}} (1-p_{1|1})^{n_{10}-1} = 0$

$n_{11} (1-p_{1|1}) = n_{10} p_{1|1}$

$p_{1|1} = \frac{n_{11}}{n_{11} + n_{10}}$

Which is the maximum likelihood expression given by Arthur

Assuming a non-informative prior then notice that the expression for $P(p_{1|1} | \{n_{ij}\})$ is a beta distribution (see http://en.wikipedia.org/wiki/Beta_distribution) with $\alpha = n_{11}+1$ and $\beta=n_{10}+1$

The mode of the beta distribution is the expression as given by Arthur, but you could put error bounds (strictly a credible interval) on this using percentiles of the beta distribution. The beta CDF is a not algebraically friendly, but R will calculate the credible interval like so:

c(qbeta(0.025, n11+1, n10+1), qbeta(0.975, n11+1, n10+1))

This also suggests that you could just as validly use the mean or median of the Beta distribution as your estimate, and in some situations that might be better.

Frequentist approach

The above is a little bit Bayesian, and so there is another approach.

The $n_{11}$ value is the number of time B=1 when A=1, while the $n_{10}$ is the number of time B=0 when A=1. Assuming independence, then the number of B=1 events, given A=1, should follow a binomial distribution with $n=n_{11}+n_{10}$ and $p=p_{1|1}$. We therefore need to estimate the $p$ parameter from the binomial distribution.

Using method of moments, we see that

$E[k] = np = (n_{11}+n_{10}) p_{1|1} $

$ n_{11} = (n_{11}+n_{10}) p_{1|1} $

$ p_{1|1} = \frac{n_{11}}{n_{11}+n_{10}} $

Recovering the same expression as before.

We could now construct a 95% confidence bound for $p_{1|1}$ by finding values of p. Since you have 10,000 rows, we can probably get away with the Normal approximation here, and so our symmetric approximate error bounds are

$\pm 1.96 \sqrt{\frac{p_{1|1}(1-p_{1|1})}{n_{01}+n_{11}}} = \pm 1.96 \sqrt{\frac{n_{11}n_{01}}{(n_{11}+n_{01})^3}} $

Answer 2

For $i,j \in \{0,1\}$, let $n_{i,j}$ be the count of the number of rows in which $A=i$ and $B=j$. Define $p_{i,j}$ by

$p_{i,j}= \frac{n_{i,j}}{n_{i,0}+n_{i,1}} .$

Then $p_{i,j}$ is the maximum likelihood estimator for $P(B =j \mid A=i)$.