# Covariance between two binomial random variables

Consider a binomial random variable $$X$$ with parameter $$p$$ and another binomial random variable $$Y$$ with parameter $$q$$. What is the covariance of $$X$$ and $$Y$$?

How well does the proof generalize to $$n$$ such random variables $$X_1...X_n$$ with respective parameters $$p_1...p_n$$? Has the covariance matrix been worked out in this case as well?

• Binomial distribution or Bernoulli distribution? If binomial, $n$ is needed. My answer is based on Bernoulli, If you mean Binomial, I will delete my answer. – user158565 Jul 14 at 5:53

Suppose the values for $$X$$ and $$Y$$ following Bernoulli distribution are 0 and 1, and $$p$$ and $$q$$ are probabilities of being 1. We know $$Cov(Y,Y) = E(XY)-E(X)E(Y)$$ and $$E(X)E(Y)=pq$$. For $$E(XY)$$, we have $$Max(E(XY)) = min(p,q)$$ and $$Min(E(XY)=min(0,p+q-1)$$ So $$min(0,p+q-1) - pq \le Cov(X,Y) \le min(p,q) -pq$$ For the situation of more than 2 random variables, it maybe is complicated.

• Do you intend for $X$ and $Y$ to be Bernoulli random variables? – BruceET Jul 14 at 5:44
• Yes. May I misunderstand the OP. That no $N$s were given mislead me to bernouli. Maybe I should delete it. – user158565 Jul 14 at 5:47
• No, please don't delete on account of my comment. Your interpretation is reasonable. But it is customary to say 'Bernoulli' when $n = 1,$ so I tried to make what sense I could of general Binomials with $n > 1.$ At least your interpretation led to an interesting answer (+1), I'm not sure I can say the same for mine. – BruceET Jul 14 at 6:04
• If OP means binomial, my answer really does not answer the question. – user158565 Jul 14 at 6:07
• It you wait for questions to be perfectly clear, there will be many fewer questions to answer. // Just browsed a few of your other answers. Some really nice ones there. – BruceET Jul 14 at 6:17

If the binomial random variable are independent, then of course the population correlation is $$0.$$ Samples from the distributions of the two random variables will tend to be near $$0.$$

set.seed(1234)
x = rbinom(10^5, 10, .3);  y = rbinom(10^5, 10, .7)
cor(x,y)
[1] -0.0006200541


However, one can simulate mixtures of binomial random variables that are correlated. In the plot below, a small amount of uniform noise keeps points that would have had exactly integer coordinates from plotting exactly on top of each other. [This is called 'jittering'.]

set.seed(2019)
p = rbeta(10^4, 2, 2)  # different p's for each (x,y)-pair below
x = rbinom(10^4, 10, p);  y = rbinom(10^4, 10, p)
cor(x, y)
[1] 0.7146091
cov(x,y)
[1] 5.03711
X = x + runif(10^4, -.2, .2)
Y = y + runif(10^4, -.2, .2)
plot(X, Y, pch=".")


Finally, if the success probabilities $$p$$ are the same in three independent binomials $$U, V,$$ and $$W,$$ each distributed $$\mathsf{Binom}(n=5, p = 0.3),$$ then $$X = U+V$$ and $$Y = U+W$$ are correlated random variables, each with distribution $$\mathsf{Binom}(10, 0.3).$$ Also,

$$Cov(X,Y) = Cov(U+V, U+W) = Cov(U,U)\\ = Var(U) = 5p(1-p) = 1.05,$$ where the second inequality is due to the mutual independence of $$U,V,$$ and $$W.$$

set.seed(713);  m = 30000
u = rbinom(m, 5, .3);  v = rbinom(m, 5, .3);  w = rbinom(m, 5, .3)
x = u + v;  y = u + w
cor(x, y);  cov(x, y)
[1] 0.5082363
[1] 1.065182   # aprx Cov(X,Y) = 1.05

par(mfrow=c(1,3));  cutp = (-1:10)+.5
k = 0:10;  pdf = dbinom(k, 10, .3)
hist(x, prob=T, br=cutp, col="skyblue2")
points(k, pdf, col="red", pch=19)
hist(y, prob=T, br=cutp, col="skyblue2")
points(k, pdf, col="red", pch=19)
X = x + runif(m, -.2, .2)
Y = y + runif(m, -.2, .2)
plot(X,Y, pch=".")
par(mfrow=c(1,1))


The first two panels show histograms of simulated marginal distributions of $$X$$ and $$Y,$$ with red dots showing exact PDFs. The simulated marginal distribution is shown (slightly jittered) in the third panel.