Let $X_1$,$X_2$...be i.i.d. with mean 1 and variance 1. Let $\epsilon_1$,$\epsilon_2$ be i.i.d with mean 0 and variance $\sigma^2$, independent of the $X_i$. Let $Y_i$=$\theta$$X_i$+$\epsilon_i$ for all i. $X_i$ and $Y_i$ are observed, but $\epsilon_i$ are unobservable and $\theta$ is unknown.

What is the covariance of ($X_i$,$Y_i$)?


So $E(Y_i)=E[\theta X_i+\epsilon_i]=\theta$

$Var(Y_i)=Var[\theta X_i+\epsilon_i]=\theta^2 +\sigma^2$

Then I use the Covariance formula and lost at how to compute $Cov=E(X_i *(\theta$$X_i+\epsilon_i))-E(X_i)E(Y_i)$


You just need the the following:$$E[X_i(\theta X_i+\epsilon_i)]=\theta E[X_i^2]+E[X_i\epsilon_i]=\theta E[X_i^2]+E[X_i]E[\epsilon_i]=\theta E[X_i^2]$$

And, $E[X_i^2]=\operatorname{var}(X_i)+E[X_i]^2=2$. You have $E[X_i],E[Y_i]$ calculated, just substitute.

  • $\begingroup$ yeah, I see that now. thanks $\endgroup$ – Iwishworldpeace Feb 16 at 19:33

Covariance is a bilinear function, meaning that $$\operatorname{cov}(aX+bY, cW+dZ) = ac\operatorname{cov}(X,W)+ad\operatorname{cov}(X,Z)+bc\operatorname{cov}(Y,W)+bd\operatorname{cov}(Y,Z).$$ Ignoring the subscripts $i$ which seem to have nothing to do with the matter at hand, we have that \begin{align} \operatorname{cov}(X,\theta X+\varepsilon) &= \theta\operatorname{cov}(X,X) + \operatorname{cov}(X,\varepsilon)\\ &= \theta\operatorname{var}(X) + 0\\ &= \theta \end{align} where we have recognized that the covariance of $X$ with itself is just another name for the variance of $X$ (which is given to be $1$) and that $X$ and $\varepsilon$ are independent random variables and thus have zero covariance. No muss, no fuss, no dragging in unnecessary computations of means etc.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.