I am new in ML so excuse me if this is a bit basic.
I noticed many times that the requirement for some methods in ML is that the instances are IID(e.g. Stochastic Gradient Descent).
I don't exactly understand this. Does this means that all features must have the same distribution and must be independent? Or does this mean that chosen instances must be independent and have the same distribution (I don't know how better to write this, not sure it is clear). And to assure IID random selection of instances from training set is enough (Because every feature has some distribution and with random selection you provide independent instance)?
