Is there a case when we assume a random variable $\epsilon$ to be IID and assume its distribution is not gaussian?

  • $\begingroup$ Welcome to CV! What kind of answer are you looking for? Different distributions have different logical substantiations. While approximate normality can often be argued for, there are few examples where it does makes sense to assume exact normality. $\endgroup$ – Frans Rodenburg Sep 10 at 2:24
  • $\begingroup$ Thank you Frans. I'm actually curious let's say in the domain of machine learning, is there a case where we assume that the errors follows a distribution other than gaussian? $\endgroup$ – Benj Cabalona Jr. Sep 10 at 2:28
  • $\begingroup$ Are you familiar with the generalized linear model? For example, for counts you assume a discrete, non-negative error distribution (Poisson, negative binomial), for ratios you assume a binomial error distribution, for time between events you assume a non-negative continuous distribution (exponential, Weibull). $\endgroup$ – Frans Rodenburg Sep 10 at 2:32
  • 2
    $\begingroup$ The description i.i.d. applies to sets of random variables, not a single random variable. It means the variables are independent and identically distributed. It doesn't imply any particular form for the distributions (Gaussian or otherwise). For example, a series of coin flips would be modeled as a set of i.i.d. Bernoulli variables. $\endgroup$ – user20160 Sep 10 at 2:37
  • $\begingroup$ It happens may times that we assume that the distribution of e is not Gaussian: every time that empirically and/or theoretically that variable is better described by another pdf. For example (but this is just an example!), it may happen that some residuals are skewed (so asymmetric), therefore you want to take into account that asymmetry that is not of a normal. Or that those residuals have fatter/thinner tails than a normal. It depends on the application, and the specific random variable that you are modeling. Many times, it will just be the empirical evidence to suggest a proxy for the pdf. $\endgroup$ – Fr1 Sep 10 at 3:01

IID random variables are not always Gaussian

The acronym IID means "independent and identically distributed". It refers to a property of a sequence of random variables, whereby those random variables are mutually independent, with a common marginal distribution. If the sequence $\epsilon_1, \epsilon_2, \epsilon_3, ... \sim \text{IID Dist}$ then we have:

$$\mathbb{P}(\epsilon_1 \leqslant e_1, \epsilon_2 \leqslant e_2, ... , \epsilon_n \leqslant e_n) = \prod_{i=1}^n F(e_i),$$

where $F$ is the (common) marginal distribution function for each of the random variables in the sequence. The distribution function $F$ can be from a Gaussian distribution, or it can be from any other distribution. There are various contexts in which we use models with IID Gaussian error terms, and in other models, we might use IID error terms that are not Gaussian. The latter are common in financial models, where the analyst generally wants to have fat tails in the distribution, in order to avoid underestimating the probability of extreme events.


Since you call it $\epsilon$, I assume you mean an error term in some kind of regression. In that case, imagine a classifier that inputs photographs and outputs the classification of dog or cat. The classifier makes some guess, based on the photo. Then there is some error term. Some photos of dogs just look like cat photos, and some cat photos look like dog photos.

Those error terms have Bernoulli distributions, so they are not Gaussian, but I am as content to consider each error independent of the others with an equal chance of making a mistake for each classification, so the error terms are iid.

  • $\begingroup$ Thank you Dave. This is exactly what i'm curious about. Im going over Stanford's CS229. Thanks again. $\endgroup$ – Benj Cabalona Jr. Sep 10 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.