2
$\begingroup$

I am "new" to probability/statistics and I was hoping someone could verify that this is correct. Let $Y_1,\ldots,Y_n$ be random variables that follow a common distribution with mean $\mu$ and variance $\sigma^2$. However, I am not assuming that the variables are independent. I want to compute the expected value and variance of $X=Y_1+\cdots+Y_n$. Then, $$\mathbb{E}(X) = \mathbb{E}\left(\sum_{i=1}^{n} Y_i \right) = \sum_{i=1}^{n} \mathbb{E}\left( Y_i \right) = n \cdot \mu.$$ Moreover, \begin{eqnarray*} \operatorname{Var}(X) &=& \operatorname{Var}\left(\sum_{i=1}^{n} Y_i \right) \\ &=& \sum_{i=1}^{n} \operatorname{Var}\left( Y_i \right) + 2\cdot\left(\sum_{1\leq i<j\leq n} \operatorname{Cov}(Y_i,Y_j) \right) \\ &=& \sum_{i=1}^{n} \operatorname{Var}\left( Y_i \right) + 2\cdot\left(\frac{ n(n -1 )}{2}\right) \cdot C \\ &=& n \cdot \sigma^2 + n(n -1)\cdot C\\ &=& n\cdot ( \sigma^2 + (n-1)\cdot C), \end{eqnarray*} where we have used the properties of the variance, and the fact that for any $i\neq j$, the variables $Y_i,Y_j$ follow the same distribution, and so $\operatorname{Cov}(Y_i,Y_j)=C$ for all $i\neq j$, for some constant $C$.

Question 1: Is this right? I have an uneasy feeling about assuming that all the covariances equal a constant C. Is there some additional hypothesis on the original random variables $Y_1,\ldots,Y_n$ that one needs to assume so that the covariance $\operatorname{Cov}(Y_i,Y_j)=C$ independent of the chosen $i\neq j$?

Question 2: Also, is $C=\operatorname{Cov}(Y_i,Y_j)=\operatorname{Cov}(Y_i,Y_i)=\sigma^2$?

Thanks!

$\endgroup$
  • $\begingroup$ Your second question asks whether all the variables are perfectly correlated. What do you think? $\endgroup$ – whuber Sep 2 '16 at 16:06
  • $\begingroup$ Your question asks whether it is possible to have Cov$(Y_i, Y_j)$ to have the same value $C$ for all $i \neq j$ where the $Y_i$ are identically distributed with variance $\sigma^2$. The answer is Yes, and the answers to this question show that it must be the case that $C \in \left[-\frac{\sigma^2}{n-1}, \sigma^2\right]$. Also, identical distribution (or same value for mean) are not required; same value for variance is all that is needed. $\endgroup$ – Dilip Sarwate Sep 21 '16 at 16:32
1
$\begingroup$

Let $n = 3$. $Cov(Y_1,Y_2), Cov((Y_1,Y_3), Cov(Y_2,Y_3)$ can all be the same or different. They can take any values, so long as the 3 by 3 covariance matrix is positive semi-definite.

A very common case, much studied on this board, would be a 3 dimensional Normal, which in your case of identically distributed components would have equal elements on the diagonal. The covariances between the components (and in this case, therefore also the correlations between the components) can be the same or different, as long as the covariance matrix is positive semi-definite.

As @whuber has hinted in a comment, while the scenario in your question 2 would be possible, and corresponds to perfectly correlated variables, which corresponds to a covariance matrix in which all entries are equal (and therefore the covariance matrix is singular), it certainly need not be the case.

$\endgroup$
1
$\begingroup$

Question 1

The additional assumption you need so that the covariances become equal is called `Exchangeability'. The definition of exchangeability is as follows:

Let $π$ denote an arbitrary permutation on n elements (that is, a function that maps $\{1, . . . , n\}$ onto itself). The finite sequence $X_1, . . . , X_n$ is said to be exchangeable if the joint distribution of the permuted random vector $(X_{π(1)}, . . . , X_{π(n)})$ is the same no matter which π is chosen. The infinite sequence X1, X2, . . . is said to be exchangeable if any finite subsequence is exchangeable.

Under exchangeability, the covariance between $X_i$ and $X_j$ is always the same, say $C$, when i $\neq$ j. Under the assumption of exchangeability, your computations are correct.

Question 2

Under exchangeability, this is not true. You will need a completely different assumption to make the covariance equal to the variance.

Check out this fragment of a book. They explain identically distributed by not identical with an example very similar to yours (instead of sum of random variables, they work with with the mean): http://sites.stat.psu.edu/~dhunter/asymp/lectures/p51to58.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.