An urn containing $pN$ white and $qN$ black balls, the total number of balls being $N$. Balls are drawn one by one without being returned to the urn until a certain number $n$ of balls is reached.

Let $ X_i= \begin{cases} 1&\text{if the i$_{th}$ drawn ball is white }\ \\ 0&\text{if the i$_{th}$ drawn ball is black}\ \end{cases} $

Show that the co-variance between $X_j$ and $X_k$ is $\dfrac {-pq}{N-1}, j \ne k$

Attempt: The joint probability density function of $X_j X_k$ can be computed as :

$\begin{array}{|c|c|c|c|c|} \hline Y=X_jX_k& (1\times1=1) & (1\times0=0) & (0\times1=0) & (0\times0=0) \\ \hline P(Y)& p^2& pq&pq &q^2\\ \hline \end{array}$

The co-varirance between $X_j \text{ and}\ X_k = E [ ~\{X_j - E(X_j) \} \{ X_k-E(X_k)\}~] = E(X_j X_k)-E(X_j)E(X_k)\\ = 1 \times p^2 - [p \times p ] = 0 $

Where could I be going wrong? Is there a conceptual error somewhere?

Thanks a lot for the help


2 Answers 2


Your calculations assume draws with replacement and therefore yields uncorrelated $X_j$ and $X_k$. First of all, $E[X_j]$ is trivial and equal to $p$ as you've also noted. We need to find $E[X_jX_k]=P(X_j=1,X_k=1)$.

Here, the key property is $$P(X_j=1,X_k=1)=P(X_1=1,X_2=1)=\frac{pN}{N}\frac{pN-1}{N-1}$$ (to see it, start with $P(X_1=1,X_3=1)=\sum_{x_2\in\{0,1\}}P(X_1=1,X_2=x_2,X_3=1)$)

Which yields $\operatorname{cov}(X_j,X_k)=p\frac{pN-1}{N-1}-p^2=\frac{-pq}{N-1}$

  • $\begingroup$ Thanks a lot. Could you please explain the last step only. Thanks again $\endgroup$
    – MathMan
    Oct 24, 2019 at 20:58
  • 1
    $\begingroup$ last step is the covariance (as well as your) formula, i.e. $E[X_jX_k]-E[X_j]E[X_k]$ $\endgroup$
    – gunes
    Oct 24, 2019 at 20:59
  • $\begingroup$ okay. So, if I have understood correctly: The general formula to prove this should be: $P(X_j=1,X_k=1)=\sum_{x \in \{0,1\}} P(X_1=x, \cdots,X_{j-1}=x,X_j=1,X_{j+1}=x,\cdots,X_{k-1}=x, X_k=1),$. Am I correct? $\endgroup$
    – MathMan
    Oct 24, 2019 at 21:09
  • 1
    $\begingroup$ No, it should be $X_1=x_1,X_2=x_2,...$, not $x$. You'll see that for a given assignment of variables inside the summation, you can permute the variables any way you like w/o disturbing number of $1$'s, which leaves us with $P(X_1=1,X_2=1)$ $\endgroup$
    – gunes
    Oct 24, 2019 at 21:11
  • 1
    $\begingroup$ @MathMan yes. And, it'll be quite clear if you start from $X_1,X_3$ case I mentioned above. You'll see that you can permute the numerator terms when number of 1's is constant which will in turn mean that you can actually permute the variables, e.g. $$P(X_1=1,X_2=0,X_3=1)=P(X_1=1,X_2=1,X_3=0)$$ $\endgroup$
    – gunes
    Oct 25, 2019 at 10:40


Suppose that there are $N=4$ balls in the urn---one white and three black. We take $n=3$ balls from the urn without replacement. We are interested in random variables $X_1,$ which takes the value 1 when the first ball is white and 0 otherwise and $X_3,$ which takes the value 1 when the third ball is white and 0 otherwise. Intuitively, it is clear that $X_1$ and $X_3$ are negatively associated because choosing a white ball on the first draw makes it impossible to get a white ball on the third.

Then in @gunes' Answer (+1), we have $E(X_1) = E(X_3) = p = .25,$ $E(X_1X_3) = (.25)(0/4) = 0,$ and $Cov(X_1,X_3) = 0 - (.25)^2 = -0.0625.$

A simulation in R of a million such 3-ball draws gives results that agree with the above to about 3 decimal places.

urn = c(0,0,0,1)
N=10; n=3; m=10^6; x1 = x3 = numeric(m)
for(i in 1:m) {
  x = sample(urn, n)
  x1[i] = x[1];  x3[i] = x[3] }

e.1 = mean(x1);  e.3 = mean(x3);  e.13 = mean(x1*x3)
e.1;  e.3;  e.13
[1] 0.249513     # aprx p = 0.25
[1] 0.249795     # aprx p = 0.25
[1] 0

e.13 - e.1*e.3
[1] -0.0623271   # aprx cov = -0.0625 from above
cov(x1, x3)
[1] -0.06232716  # aprx cov = -0.0625 using R fcn

A key point is that $0 = P(X_1 = 1, X_3 = 1) \ne P(X_1 = 1)P(X_3 = 1) = 0.0625.$

mean(x1==1 & x3==1)
[1] 0
mean(x1==1)* mean(x3==1)
[1] 0.0623271     # aprx 0.0625

One way to show this in a plot is to choose 10,000 of the 1 million points simulated, and jitter them (with random uniform noise) to avoid overplotting. Then the probability at a point is represented by the density of points in a square with the point (red) at its center.

enter image description here

The left-hand panel shows results from the simulation above. At right, is a similar plot from balls sampled with replacement, so that draws are independent.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.