Why it is often assumed Gaussian distribution? Quoting from a Wikipedia article on parameter estimation for a naive Bayes classifier: "a typical assumption is that the continuous values associated with each class are distributed according to a Gaussian distribution."
I understand that a Gaussian distribution is convenient for analytical reasons. However, is there any other real-world reason to make this supposition? What if the population consists of two sub-populations (smart/dumb people, large/small apples)?
 A: At least for me, the assumption of normality arises from two (very powerfull) reasons:


*

*The Central Limit Theorem.

*The Gaussian distribution is a maximum entropy (with respect to the continuous version of Shannon's entropy) distribution.
I think you are aware of the first point: if your sample is the sum of many procceses, then as long as some mild conditions are satisfied, the distribution is pretty much gaussian (there are generalizations of the CLT where you in fact don't have to assume that the r.v.s of the sum are identically distributed, see, e.g., the Lyapunov CLT).
The second point is one that for some people (specially physicists) make more sense: given the first and second moments of a distribution, the distribution which less information assumes (i.e. the most conservative) with respect to the continuous Shannon's entropy measure (which is somewhat arbitrary on the continuous case, but, at least for me, totally objective in the discrete case, but that's other story), is the gaussian distribution. This is a form of the so called "maximum entropy principle", which is not so widespread because the actual usage of the form of the entropy is somewhat arbitrary (see this Wikipedia article for more information about this measure). 
Of course, this last statement is true also for the multi-variate case, i.e., the maximum entropy distribution (again, with respect to the continuous version of Shannon's entropy) given first ($\vec{\mu}$) and second order information (i.e., the covariance matrix $\mathbf{\Sigma}$), can be shown to be a multivariate gaussian.
PD: I must add to the maximum entropy principle that, according to this paper, if you happen to known the range of variation of your variable, you have to make adjustments to the distribution you get by the maximum entropy principle.
A: The use of the CLT for justifying the use of the Gaussian distribution is a common fallacy because the CLT is applied to the sample mean, not to individual observations. Therefore, increasing your sample size, does not mean that the sample is closer to normallity.
The Gaussian distribution is commonly used because:


*

*Maximum likelihood estimation is straightforward.

*Bayesian inference is simple (using conjugate priors or Jeffreys-type priors).

*It is implemented in most of the numerical packages.

*There is a lot of theory about this distribution in terms of hypothesis testing.

*Lack of knowledge about other options (more flexible).
...


Of course, the best option is to use a distribution that takes into account the characteristics of your context,  but this can be challenging.  However, is something that people should do 
"Everything should be made as simple as possible, but not simpler." (Albert Einstein)
I hope this helps.
Best wishes.
A: My answer agrees with the first responder.  The central limit theorem tells you that if your statistic is a sum or average it will be approximately normal under certain technical conditions regardless of the distribution of the individual samples.  But you are right that sometimes people carry this too far just because it seems convenuent.  If your statistic is a ratio and the denominator can be zero or close to it the ratio will be too heavytailed for the normal.  Gosset found that even when you sample from a normal distribution a normalized average where the sample standard deviation is used for the normalization constant the distribution is the t distribution with n-1 degrees of freedom when n is the sample size.  In his field experiments at the Guiness Brewery he has sample sizes that could be in the range of 5-10.  In those cases the t distribution is similar to the standard normal distribution in that it is symmetric about 0 but it has much heavier tails. Note that the t distribution does converge to the standard normal as n gets large.
In many cases the distribution you have might be bimodal as it is a mixture of two populations.  Some times these distributions can be fit as a mixture of normal distributions. But they certain do not look like a normal distribution.  If you look at a basic statistics textbook you will find many parametric continuous and discrete distributions that often come up in inference problems.  For discrete data we have the binomial, Poisson, geometric, hypergeometric and negative binomial to name a few.  Continous examples include the chi square, lognormal, Cauchy, negative exponential, Weibull and Gumbel.
