What references should be cited to support using 30 as a large enough sample size? I have read/heard many times that the sample size of at least 30 units is considered as "large sample" (normality assumptions of means usually approximately holds due to the CLT, ...).  Therefore, in my experiments, I usually generate samples of 30 units. Can you please give me some reference which should be cited when using sample size 30?
 A: IMO, it all depends on what you want to use your sample for.  Two "silly" examples to illustrate what I mean: If you need to estimate a mean, 30 observations is more than enough.  If you need to estimate a linear regression with 100 predictors, 30 observations will not be close to enough.
A: Actually, the "magic number" 30 is a fallacy. See Jacob's Cohen's delightful paper, Things I Have Learned (So Far) (Am. Psych. December 1990 45 #12, pp 1304-1312). This myth is his first example of how "some things you learn aren't so".

[O]ne of my fellow doctoral candidates undertook a dissertation [with] a sample of only 20 cases per group. ... [L]ater I discovered ... that for a two-independent-group-mean comparison with $n = 30$ per group at the sanctified two-tailed $.05$ level, the probability that a medium-sized effect would be labeled as significant by ... a t test was only $.47$. Thus, it was approximately a coin flip whether one would get a significant result, even though, in reality, the effect size was meaningful. ... [My friend] ended up with nonsignificant results–with which he proceeded to demolish an important branch of psychoanalytic theory.

A: The choice of n = 30 for a boundary between small and large samples is a rule of thumb, only. There is a large number of books that quote (around) this value, for example, Hogg and Tanis' Probability and Statistical Inference (7e) says "greater than 25 or 30".
That said, the story told to me was that the only reason 30 was regarded as a good boundary was because it made for pretty Student's t tables in the back of textbooks to fit nicely on one page.  That, and the critical values (between Student's t and Normal) are only off by approximately up to 0.25, anyway, from df = 30 to df = infinity.  For hand computation the difference didn't really matter.
Nowadays it is easy to compute critical values for all sorts of things to 15 decimal places.  On top of that we have resampling and permutation methods for which we aren't even restricted to parametric population distributions.
In practice I never rely on n = 30.  Plot the data.  Superimpose a normal distribution, if you like.  Visually assess whether a normal approximation is appropriate (and ask whether an approximation is even really needed).  If generating samples for research and an approximation is obligatory, generate enough of a sample size to make the approximation as close as desired (or as close as computationally feasible).
A: This is meant to supplement user1108's answer stating that:

That said, the story told to me was that the only reason 30 was regarded as a good boundary was because it made for pretty Student's t tables in the back of textbooks to fit nicely on one page. That, and the critical values (between Student's t and Normal) are only off by approximately up to 0.25, anyway, from df = 30 to df = infinity. For hand computation the difference didn't really matter.

I did some investigation on this issue, and the earliest source I can find is Fisher's Statistical Methods for Research Workers (1925). I remember examining a copy of this text (you can see http://psychclassics.yorku.ca/Fisher/Methods/, for example) and noticing that the following table neatly fit on one page.

From what I recall reading in the text, there is nothing justifying why Fisher chose to stop at $n = 30$. So as far as I know, the only justification for this is that such tables can fit neatly on one page back in the day.
A: Mostly arbitrary rule of thumb. This statement depends on a number of factor to be true. For example on the distribution of the data. If the data comes from a Cauchy for example, even 30^30 observations are not enough to estimate the mean (in that case even an infinite number of observations would not be enough to cause $\bar{\mu}^{(n)}$ to converge). This number (30) is also false if the values you draw are not independent from one another (again, you may have that there are no convergence at all, regardless of sample size).
More generally, the CLT needs essentially two pillars to hold:


*

*That the random variables are independent: that you can re-order your observations without losing any information*.

*That the r.v. come from a distribution with finite second moments: meaning that the classical estimators of mean and s.d. tend to converge as sample size increases.


(Both these condition can be somewhat weakened, but the differences are largely of theoretical nature)
