I was reading about Maximum likelihood estimation from various sources on the internet and I noticed that MLE makes an assumption about the data known as IID but I didn't completely understand why is it necessary to make this assumption? Are there any other assumptions that MLE makes?

  • 4
    $\begingroup$ It isn't necessary. MLE maximizes a wide array of likelihoods. Your sources are assuming iid because it is a reasonable modeling assumption in a particular setting. If your model assumes the data are iid, it is easy to write down the likelihood and then use MLE to estimate the unknown parameters. $\endgroup$
    – Taylor
    Jan 8, 2019 at 23:38
  • 3
    $\begingroup$ ... it's reasonable and it's extremely convenient, because it implies that the overall likelihood of the data set is the product of the likelihoods of the individual observations (independent), which all have the same form (identically distributed) $\to$ log-likelihood is the sum of individual LLs ... $\endgroup$
    – Ben Bolker
    Jan 8, 2019 at 23:40
  • 1
    $\begingroup$ @Taylor I didn't quite understand 'wide array of likelihoods'. Can you name a few? $\endgroup$ Jan 8, 2019 at 23:40
  • 3
    $\begingroup$ Consider a time series model like an AR(1) process. Then the order of the data matter, and your likelihood is a product of conditional densities, not marginal densities. If you have three data points, you could write the likelihood as $p(x_3 \mid x_2) p(x_2 \mid x_1) p(x_1)$. MLE would have no problem with this, but we are not assuming the data are iid. $\endgroup$
    – Taylor
    Jan 8, 2019 at 23:44
  • 2
    $\begingroup$ Possible duplicate of What does it mean by independently and identically distributed random variables? $\endgroup$
    – Taylor
    Jan 8, 2019 at 23:45

1 Answer 1


Assuming independence is not necessary for maximum likelihood estimation (ML) (or other likelihood based methods). But if independence is a reasonable assumption, then it makes ML easy to implement, since the log likelihood is simply the sum of the individual log likelihoods. But there are lots of examples where ML is used without Independence: time series with ARMA (or ARIMA) models, spatial models including spatial dependence, mixed models where there are correlation between observations within some groups, others.

When is it reasonable to assume independence? is discussed already multiple times on this site, see for example Are "random sample" and "iid random variable" synonyms? or Independence of events in real-life data

  • $\begingroup$ How does it makes ML easy to implement. Can you edit your answer and explain how it makes the maths easy? $\endgroup$ Jan 9, 2019 at 23:41
  • $\begingroup$ Will try tomorrow! $\endgroup$ Jan 9, 2019 at 23:42
  • $\begingroup$ Can you share any link for now? Or any reference or book? $\endgroup$ Jan 9, 2019 at 23:43
  • $\begingroup$ It’s worth pointing out that the theory for MLE is easiest to develop for iid data. So it makes sense for textbooks to focus on this case. $\endgroup$
    – guy
    Jan 10, 2019 at 0:05
  • 1
    $\begingroup$ @guy I tired looking for the same on the internet but couldn't get any. It would be great if you can share any references for the same. $\endgroup$ Jan 10, 2019 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.