Covariance of X and Y

Question

Let $X_1$,$X_2$...be i.i.d. with mean 1 and variance 1. Let $\epsilon_1$,$\epsilon_2$ be i.i.d with mean 0 and variance $\sigma^2$, independent of the $X_i$. Let $Y_i$=$\theta$$X_i$+$\epsilon_i$ for all i. $X_i$ and $Y_i$ are observed, but $\epsilon_i$ are unobservable and $\theta$ is unknown.

What is the covariance of ($X_i$,$Y_i$)?

Cov($X_i$,$\theta$$X_i$+$\epsilon_i$)

So $E(Y_i)=E[\theta X_i+\epsilon_i]=\theta$

$Var(Y_i)=Var[\theta X_i+\epsilon_i]=\theta^2 +\sigma^2$

Then I use the Covariance formula and lost at how to compute $Cov=E(X_i *(\theta$$X_i+\epsilon_i))-E(X_i)E(Y_i)$

Answer 1

You just need the the following:$$E[X_i(\theta X_i+\epsilon_i)]=\theta E[X_i^2]+E[X_i\epsilon_i]=\theta E[X_i^2]+E[X_i]E[\epsilon_i]=\theta E[X_i^2]$$

And, $E[X_i^2]=\operatorname{var}(X_i)+E[X_i]^2=2$. You have $E[X_i],E[Y_i]$ calculated, just substitute.

Answer 2

Covariance is a bilinear function, meaning that $$\operatorname{cov}(aX+bY, cW+dZ) = ac\operatorname{cov}(X,W)+ad\operatorname{cov}(X,Z)+bc\operatorname{cov}(Y,W)+bd\operatorname{cov}(Y,Z).$$ Ignoring the subscripts $i$ which seem to have nothing to do with the matter at hand, we have that \begin{align} \operatorname{cov}(X,\theta X+\varepsilon) &= \theta\operatorname{cov}(X,X) + \operatorname{cov}(X,\varepsilon)\\ &= \theta\operatorname{var}(X) + 0\\ &= \theta \end{align} where we have recognized that the covariance of $X$ with itself is just another name for the variance of $X$ (which is given to be $1$) and that $X$ and $\varepsilon$ are independent random variables and thus have zero covariance. No muss, no fuss, no dragging in unnecessary computations of means etc.