I always read random variables as being independent and identically distributed. I understand the concept of being identically distributed, because if different random variables are distributed in more ways than one, then getting a pdf of a combination of random variables would be extremely difficult.

But what about 'independence'? Why do we always say: Let $X_1, X_2, X_3, \cdots, X_n$ be iid?

What would happen if $X_1, X_2, X_3, \cdots, X_n$ are NOT independent but identically distributed?

Your insights would be great.

  • 1
    what do you mean by "a combination" there? There are many ways to combine variables. – Glen_b Apr 18 '15 at 10:50

iid is generally used to describe a sample $S=\{x_1,\dots,x_n\}$ of random variables $X_1,\dots,X_n$. Being independent refers to how the (physical) process of collecting the sample was performed and it assures the representative fairness of the sampling. Dependent samples introduce bias into the results, e.g. if on a voting-intention study both, husband and wife, were asked about their voting intention, we would get, high likely, dependent answers; thus, miss-representing the population.

From a computational point of view, independency significantly simplifies operations due to the factorization $P(X_1\leq x_1,\dots,X_n\leq x_n)=P(X_1\leq x_1)\cdots P(X_n\leq x_n)$, avoiding a possibly curse on dimensionality. For instance, consider the painful computation of a maximum-likelihood estimator based on a dependent sample.

However, it is not always the case that we are interested in iid samples. Times series analysis is based on the dependency among the random variables, as many other stochastic processes are. Dependence and conditional dependence structures are at the core of models such as markov chain and, more generally, graphical models.

$X_1, X_2, X_3, \cdots, X_n$

That's okay if you say non-identical distribution are difficult to combine, linear or product, but you can't combine them easily if RVs are dependent.

Independence of RVs is more important than identical, you can combine independent and non-identical distribution see. But you can't combine easily for non-independent and identical distributions.

as convolution demand for independent RVs, and

Joint probability distribution for independent RVs is just product of RVs, otherwise you need to calculate from start.you can find Joint probability distribution for 2 dependent RVs. But if you go for more non-independent RVs you shall follow Chain Rule which will be getting difficult.

Primarily, if they are not independent, the complexity of the problem explodes. Admittedly, the assumption of iid is sometimes poor and the complexity is then justified.

Indeed, if you take treat time of discovery as a measure of complexity, typical linear regression has been around for ages, and only works properly if there is independence between Y and the error term. Linear regression is ancient, and highly versatile otherwise. If Y is correlated with itself, say over time, then you have autocorrelation, and it becomes substantially harder and the development and maturation of time series is far far later.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.