# Independent and Identically Distributed (i.i.d.) random variables

The assumption that observations are i.i.d. tends to simplify the mathematics of many statistical methods. However, the assumption may or may not be realistic in practical applications of statistical modeling. To test the assumption on a given data set, the autocorrelation is typically computed.

How does assessing autocorrelation check the assumption that the data are i.i.d? As far I know, the autocorrelation checks for independence.

• You realistically cannot test if the data observations are truly iid. You can get approximations as stated below. Also correlation is really a one way street that is if you know that rv's X1 and X2 are independent then the correlation=0. On the other hand if can show that the correlation is not equal to 0 then you have shown that the random variables are not independent( contrapositive). – user60887 Dec 16 '14 at 0:06

So, I think there are a few important facets to your question.

Let's differentiate first between autocorrelation and correlation. Autocorrelation is a property that is exhibited by a time series where lagged items have a positive correlation with the most recent observation. Most books on time series talk about autocorrelation in the context of a univariate (observations of a single variable through time). Statistical tests that indicate the presence of autocorrelation typically end up with variations of the Portmanteau statistics and their adaptations (see the Ljung-Box statistic: http://en.wikipedia.org/wiki/Ljung%E2%80%93Box_test).

An example here might be that I have a time series where the current observation is correlated with the observations 2, 5, and 7 periods ago.

Correlation determines whether two random variables have a degree of dependence, but aren't necessarily time ordered in their observations. There are ways to help determine whether two variables of non-time-series data have correlations that are statistically different from 0 (see http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Inference).

An example here might be that I have data about 100 students. I determine that there is a correlation of .4 between "GPA" and "Age".

Where I think your question might be going... Assuming that you have time ordered data, you should find no significant autocorrelation in the data and its lags. This potentially gives you a reasonable assumption of independence. For the "identically distributed" piece, you need to perform a distributional fit of the data and look for strange outlier behavior. Assuming that you know the theoretical distribution underlying the data, tests such as the Anderson Darling test, Skewness and Kurtosis tests (testing for normality), Chi Square Goodness of Fit Test (seeing if data fits a distribution based on expected frequencies), and Kolmogorov-Smirnov tests (seeing if empirical and theoretical cumulative distribution functions have significant deviations) can be used. You can also perform a distributional fit to see if a parametric distribution of some form approximates the data well enough.

If you need to determine whether two sets of data are identically distributed, I'd suggest performing tests for equality of means, variances, skewnesses, and kurtoses. I haven't encountered it yet, but you might be able to use a multivariate version of Ljung-Box to determine independence between time series (Ljung–Box test for a multivariate time series?)

In any case, all of the statistical procedures provide statements in probability, so you can never truly know if the data are 100% independently and identically distributed; however other knowledge of the processes generating the data may help you determine whether these assumptions are reasonable.