A version of the central limit theorem tells us that the sample means will be distributed roughly like a normal distribution around the population mean.
Are there cases of a sample statistics that don't behave normally? How do they behave? Are all of their distributions independent from the population distribution?
It feels like I'm barely scratching the surface of the definition of CLT. A similar question was asked here.
The answer to the title question is no.
As an example, suitably standardized sample maxima and minima have limiting distributions that are not Gaussian (see the Fisher-Tippett-Gnedenko theorem).
For another example, see the Pickands-Balkema-de Haan theorem, which is about limiting distributions of values above a threshold.
There are many examples on site of statistics that (when standardized) are not asymptotically normal.
For example, many questions on site discuss the smallest or largest observation from a sample from a uniform population (which statistics have beta distributions). An appropriately 'standardized' smallest observation from a uniform distribution on $(0,k)$ will be asymptotically exponential.
An easier example: $n$ times the smallest observation from sample of size $n$ from a standard exponential is standard exponential, so the limit in this case is trivial (being attained at every sample size).
(These two are both special cases of the Fisher-Tippett-Gnedenko theorem)
Another interesting example is the Kolmogorov-Smirnov statistic $D_n=\sup_{x}|F_n(x)-F(x)|$ (under the null hypothesis); suitably 'standardized' ($K_n=\sqrt{n}D_n$) it approaches the Kolmogorov distribution in the limit.
You might also like to consider the distribution of the sample median in a discrete distribution. For simplicity, consider the case where the population median is at a specific point rather than an interval (skirting, for example, the additional detail required for dealing with a $\text{Poisson}(n)$ for integer $n$). In the limit, the sample median will have all its probability at a single point (the population median).
