# Covariance of 2 sequences of random variables

We have a random number generator that generates random integers independently uniformly distributed from 1 to 5, inclusive. This random number generator is used to generate a sequence on $n$ independent random integers.

Let $X$ = number of $1$’s generated in the sequence of $n$ integers. Let $Y$ = number of $5$’s generated in the sequence of $n$ integers. (Hint: it might be useful to define indicator variables denoting whether the $i$-th integer generated is a 1, and likewise for whether the $i$-th integer generated is a 5.)

What is $\text{cov}(X, Y)$?

• @AlexeyGrigorev, it is best not to add the [self-study] tag for the OP. Instead, ask them to add the tag & read its wiki. Then there is some chance they will be familiar w/ our policies. May 25, 2015 at 20:33

As a response to a question, this answer aims to share a problem solving process rather than merely stating an answer.

There doesn't seem to be anything special about ones and fives: because "one" and "five" are just different labels for two of the five possible outcomes, the joint distribution of $(X,Y)$ will be identical to the joint distribution of a corresponding $X$ and $Y$ if we were to replace $1$ and $5$ by any two distinct values.

With this in mind, let's apply the hint by defining $Z^{(j)}_i = 1$ when the $i^\text{th}$ random integer has the value $j$ (and otherwise $Z^{(j)}_i=0$). Thus the number of $j$s appearing in the sequence is

$$Z^{(j)} = \sum_{i=1}^n Z^{(j)}_i.$$

Indeed, let's generalize to the case $5 = M$, where $M$ is some positive integer. In this notation $X = Z^{(1)}$ and $Y=Z^{(M)}$.

The question asks about covariances. These depend on the first two moments. Having already observed that $1$ and $5$ are not so special, we might be inspired to consider a beautifully symmetric quantity

$$Z = Z^{(1)} + Z^{(2)} + \ldots + Z^{(M)}.$$

As a random variable it's rather uninteresting: by totaling all $M$ counts, it must invariably equal $n$ But as an example of its usefulness consider its first moment,

$$n = \mathbb{E}(n) = \mathbb{E}(Z) = \mathbb{E}\left(Z^{(1)} + Z^{(2)} + \ldots + Z^{(M)}\right) \\= \mathbb{E}(Z^{(1)}) + \mathbb{E}(Z^{(2)}) + \cdots + \mathbb{E}(Z^{(M)})$$

The reasoning (about distributional equivalence) at the outset implies all $M$ of these counts must have equal expectations. Therefore

$$\frac{n}{M} = \mathbb{E}(Z^{(1)}) = \mathbb{E}(Z^{(2)}) = \cdots = \mathbb{E}(Z^{(M)}).$$

That was easy!

Let's try a second-order example by computing the variance of $Z$. This obviously is zero, because $Z$ does not vary at all, whence

$$0 = \text{Var}(Z) = \text{Var}\left(Z^{(1)} + Z^{(2)} + \ldots + Z^{(M)}\right).$$

Algebraic rules for computing variances allow this to be expressed as sums of the variances of the $Z^{(j)}$ and the covariances of the $Z^{(j)}$ and $Z^{(l)}$ (for $j\ne l$). That is promising: if we could find the variances, we could solve for the common covariance, which is the goal of this exercise.

As a final hint, note that each $Z^{(j)}$ has a Binomial$(n,1/M)$ distribution.