# Independence and conditional distribution

In a problem that I'm solving I find that:

"Let data (yi,xi) be sampled randomly from a two-dimensional distribution such that y|x is N(ɑ,x^2σ^2)".

Are y and x i.i.d? maybe just identically distributed but not independent?

Finally, if my model is: y = ɑ - u (where u is the error term and ɑ is a constant), and I also know that the distribution of x does not depend on either a or σ (I don't know it this assumption is needed for answer at the following question), is u independent from x?

My question is due to the fact that I thought that if the data are sampled randomly this mean that the r.v. are independent, but how is possible that the conditional distribution (his variance) depend then on x?

• If the distribution of $y$ explicitly depends on $x$ how can they be independent? We also certainly can't say they have the same distribution when $x$ is completely arbitrary – dsaxton Jan 1 at 21:49
• @dsaxton Therefore the fact that (yi,xi) are sampled randomly does not imply independence? – Albert Jan 1 at 21:53

You are considering a random sample of size $$N$$. Each draw $$i$$ performed independently of any other draw $$j$$. Each draw is a draw of a random vector $$(Y_i,X_i)$$. Here is an illustration of all the stochastic vectors under consideration:

$$(Y_1,X_1),(Y_2,X_2),...,(Y_N,X_N)$$

As already mentioned $$(Y_i,X_i)$$ is independent of $$(Y_j,X_j)$$ for any pair $$i \not =j$$. So $$(Y_1,X_1)$$ is independent from $$(Y_2,X_2)$$. This means that $$Y_2$$ is independent of $$Y_1$$ and $$X_1$$ and $$X_2$$ is independent of $$Y_1$$ and $$X_1$$.

$$f(Y_1,X_1,Y_2,X_2) = f(Y_1,X_1)f(Y_2,X_2)$$ integrate out $$X_1$$ and $$X_2$$ to get

$$f(Y_1,Y_2) = f(Y_1)f(Y_2)$$

But for each $$i$$ it is NOT the case that $$Y_i$$ is independent of $$X_i$$ the could be draws from multivariate normal with non-zero covariance.

Assuming further they are all drawn from the same distribution so for all $$i$$ the vector $$(Y_i,X_i)\sim(Y,X)$$, which is simply one convenient way of saying that all the vectors $$i=1,...,N$$ follows same distribution.

So we have

(1) Dependence between $$Y_i$$ and $$X_i$$ for all $$i$$

(2) Independence for $$(Y_i,X_i)$$ and $$(Y_j,X_j)$$ for all $$i\not=j$$

(3) Identical distribution $$(Y_i,X_i)\sim(Y,X)$$ for all $$i$$

Define

$$Y_i = a + X_i z_i$$

with $$z_i \sim \mathcal N(0,\sigma^2)$$

then $$Y_i \lvert X_i$$ is normal and have mean $$\mathbb E[Y_i \lvert X_i]$$ and variance $$Var(Y_i \lvert X_i)$$. Calculate the mean

$$\mathbb E[Y_i \lvert X_i] = a + X_i\mathbb E[z_i \lvert X_i] = a$$ where the last step follows from assuming that $$z_i$$ is independent of $$X_i$$ so $$\mathbb E[z_i \lvert X_i] = \mathbb E[z_i] = 0$$.

Calculate variance

$$Var(Y_i\lvert X_i) := \mathbb E[(Y_i - \mathbb E[Y_i\lvert X_i])^2 \lvert X_i] \\ \mathbb E[(X_iz_i - \mathbb E[X_iz_i\lvert X_i])^2\lvert X_i] \\ \mathbb E[(X_i^2(z_i - \mathbb E[z_i\lvert X_i])^2\lvert X_i]$$ use independence of $$z_i$$ and $$X_i$$ to get $$= \mathbb E[(X_i^2(z_i - \mathbb E[z_i])^2\lvert X_i] \\ = X_i^2 \mathbb E[(z_i - \mathbb E[z_i])^2\lvert X_i] \\ = X_i^2 \mathbb E[(z_i - \mathbb E[z_i])^2]\\ = X_i^2 \sigma^2$$

$$Y_i = a + u_i$$
then $$u_i = X_i z_i$$ and is not independent of $$X_i$$.