1
$\begingroup$

While reading an ML book, I realized that most of the time, the input data points are correlated with each other, and hence their observation is not independent. But then, why do we assume that the input data is iid?

For example, we are collecting data about house prices in a locality, the houses near to each other will have similar price for similar houses. When such data is correlated, how can we say that the data points are iid?

I possibly might have not understood something here, please let me know.

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ If you don't think that independence is a good approximation for a data set, don't assume it. $\endgroup$
    – Glen_b
    Commented Jun 24, 2015 at 11:02

2 Answers 2

Reset to default
1
$\begingroup$

Sometimes a perfectly random sampling is not possible. Your example is one of these cases. There are two major ways to deal with this :

  • Model how these points are "correlated", for example with a random effect.

  • Analyse your data at a higher level to restore i.i.d assumption. For your example, If you can investigate different random areas, one could take group of houses as reference points by meaning their features.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Actually, I got an answer to this question. In the book, iid is used in context of probabilistic distributions. As an example, if we take a coin toss experiment, we would naturally assume that subsequent trials are independent of previous ones. Likewise, any probabilistic distribution applies the same way (Even continuous ones like a Gaussian). Coin toss is an example of a multinomial distribution, in this case. $\endgroup$
    – Udaya
    Commented Jun 24, 2015 at 11:55
  • $\begingroup$ This iid assumption is an artifact of modeling a physical phenomenon as a random process. The moment we assume that a physical phenomenon is a random process, we should also be able to assume that it's observations are, in fact, iid. $\endgroup$
    – Udaya
    Commented Jun 24, 2015 at 11:57
  • $\begingroup$ Yes, I like the way your book put it. But that does not discard my answer though. If you don't model your "random process" properly (for example by not considering in the model obvious design-related effects like the one you pointed out), this is wrong and we are used to call this wrongness i.i.d violation. $\endgroup$
    – brumar
    Commented Jun 24, 2015 at 12:20
  • $\begingroup$ I agree with you. That was what my point was, actually. $\endgroup$
    – Udaya
    Commented Jun 24, 2015 at 12:25
0
$\begingroup$

Your question is a classical example in survey analysis, where respondents are generally not independent.

Consider weighting and clustering.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.