Consider a binomial random variable $X$ with parameter $p$ and another binomial random variable $Y$ with parameter $q$. What is the covariance of $X$ and $Y$?

How well does the proof generalize to $n$ such random variables $X_1...X_n$ with respective parameters $p_1...p_n$? Has the covariance matrix been worked out in this case as well?

  • 1
    $\begingroup$ Binomial distribution or Bernoulli distribution? If binomial, $n$ is needed. My answer is based on Bernoulli, If you mean Binomial, I will delete my answer. $\endgroup$ – user158565 Jul 14 at 5:53

Suppose the values for $X$ and $Y$ following Bernoulli distribution are 0 and 1, and $p$ and $q$ are probabilities of being 1. We know $$Cov(Y,Y) = E(XY)-E(X)E(Y)$$ and $$E(X)E(Y)=pq$$. For $E(XY)$, we have $$Max(E(XY)) = min(p,q)$$ and $$Min(E(XY)=min(0,p+q-1)$$ So $$min(0,p+q-1) - pq \le Cov(X,Y) \le min(p,q) -pq$$ For the situation of more than 2 random variables, it maybe is complicated.

  • $\begingroup$ Do you intend for $X$ and $Y$ to be Bernoulli random variables? $\endgroup$ – BruceET Jul 14 at 5:44
  • $\begingroup$ Yes. May I misunderstand the OP. That no $N$s were given mislead me to bernouli. Maybe I should delete it. $\endgroup$ – user158565 Jul 14 at 5:47
  • $\begingroup$ No, please don't delete on account of my comment. Your interpretation is reasonable. But it is customary to say 'Bernoulli' when $n = 1,$ so I tried to make what sense I could of general Binomials with $n > 1.$ At least your interpretation led to an interesting answer (+1), I'm not sure I can say the same for mine. $\endgroup$ – BruceET Jul 14 at 6:04
  • $\begingroup$ If OP means binomial, my answer really does not answer the question. $\endgroup$ – user158565 Jul 14 at 6:07
  • $\begingroup$ It you wait for questions to be perfectly clear, there will be many fewer questions to answer. // Just browsed a few of your other answers. Some really nice ones there. $\endgroup$ – BruceET Jul 14 at 6:17

If the binomial random variable are independent, then of course the population correlation is $0.$ Samples from the distributions of the two random variables will tend to be near $0.$

x = rbinom(10^5, 10, .3);  y = rbinom(10^5, 10, .7)
[1] -0.0006200541

However, one can simulate mixtures of binomial random variables that are correlated. In the plot below, a small amount of uniform noise keeps points that would have had exactly integer coordinates from plotting exactly on top of each other. [This is called 'jittering'.]

p = rbeta(10^4, 2, 2)  # different p's for each (x,y)-pair below
x = rbinom(10^4, 10, p);  y = rbinom(10^4, 10, p)
cor(x, y)
[1] 0.7146091
[1] 5.03711
X = x + runif(10^4, -.2, .2)
Y = y + runif(10^4, -.2, .2)
plot(X, Y, pch=".")

enter image description here

Finally, if the success probabilities $p$ are the same in three independent binomials $U, V,$ and $W,$ each distributed $\mathsf{Binom}(n=5, p = 0.3),$ then $X = U+V$ and $Y = U+W$ are correlated random variables, each with distribution $\mathsf{Binom}(10, 0.3).$ Also,

$$Cov(X,Y) = Cov(U+V, U+W) = Cov(U,U)\\ = Var(U) = 5p(1-p) = 1.05,$$ where the second inequality is due to the mutual independence of $U,V,$ and $W.$

set.seed(713);  m = 30000
u = rbinom(m, 5, .3);  v = rbinom(m, 5, .3);  w = rbinom(m, 5, .3)
x = u + v;  y = u + w
cor(x, y);  cov(x, y)
[1] 0.5082363
[1] 1.065182   # aprx Cov(X,Y) = 1.05

par(mfrow=c(1,3));  cutp = (-1:10)+.5
k = 0:10;  pdf = dbinom(k, 10, .3)
 hist(x, prob=T, br=cutp, col="skyblue2")
  points(k, pdf, col="red", pch=19)
 hist(y, prob=T, br=cutp, col="skyblue2")
  points(k, pdf, col="red", pch=19)
X = x + runif(m, -.2, .2)
Y = y + runif(m, -.2, .2)
 plot(X,Y, pch=".")

The first two panels show histograms of simulated marginal distributions of $X$ and $Y,$ with red dots showing exact PDFs. The simulated marginal distribution is shown (slightly jittered) in the third panel.

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.