CLT may fail under this condition? CLT says that when sample size n goes to infinity, the sample sum or average curve converges to normal distribution.
My question is: if the sample size goes closer or even equal to the population size, wouldn't the average values cram so closely near the mean, that violate the "bell shape" distribution?
 A: If you're going to run an asymptotic argument with $n$ close to $N$, you need sequences of finite populations and samples.  For each $m=1,2,3,\dots$ suppose you have a population of size $N_m$ and a sample of size $n_m$, with $N_m\geq n_m$ and $n_m\to\infty$.  We'll need some assumptions about the populations; we can go back and work out what was needed after the argument
We know already (under reasonable assumptions) that if $n_m\to\infty$ we have a Central Limit Theorem, either with $n_m/N_n\to 0$ or with $n_m/N_n\to c\in(0,1)$. If $\mu_m$ is the true finite population mean and $\sigma^2$ is the limit of the true population variances then
$$\sqrt{n_m}(\bar X_{n_m}-\mu_m)\stackrel{d}{\to} N(0,\sigma^2)$$
Suppose $n_m$ is really big, so that the number of unsampled individuals does not go to infinity.  In that case the CLT does fail. In the extreme case, suppose $n_m=N_m-1$, so that only one individual is not sampled.  We could try to get a scaled mean to be Normal in a few ways
First, we could try $\sqrt{n_m}(\bar X_{n_m}-\mu_m)$. That doesn't work because there's only one observation's difference between $\bar X_{n_m}$ and $\mu_m$: we get $$\sqrt{n_m}(\bar X_{n_m}-\mu_m)=\sqrt{n_m}\cdot O_p(1/n_m)\stackrel{p}{\to}0$$
We could rescale and try $n_m(\bar X_{n_m}-\mu_m)$. That's of the right order, but it is just equal $X_{n_m}-\mu$, the one unsampled observation centered minus the true mean.  Or we could scale by $\sqrt{N_m-n_m}=1$, the square root of the unsampled size, but that goes to zero. Or scale the total to get
$$\frac{N_m}{\sqrt{N_m-n_m}}(\bar X_{n_m}-\mu_m)$$
which is again just the unsampled observation minus the mean.
If $N_m-n_m$ is bigger than one but bounded, you get the same sort of result: depending on the scaling it blows up, goes to zero, or gives you a finite sum that doesn't converge to Normal.
However, if $N_n-n_m\to\infty$ you are good. The unsampled observations are an infinite sum that then does satisfy a CLT, and
$$\frac{1}{\sqrt{N_m-n_m}}\left(\sum_{i=1}^{n_m} (X_i-\mu_m)\right)\stackrel{d}{\to}N(0,\sigma^2)$$
so
$$\frac{n_m}{\sqrt{N_m-n_m}}\left(\bar X_{n_m}-\mu_m\right)\stackrel{d}{\to}N(0,\sigma^2)$$
It's still true, though, that
$\sqrt{n_m}(\bar X_{n_m}-\mu_m)$
will blow up.
Ok, so what did we need to assume? I think it would suffice that the population variances $$\sigma^2_m=\frac{1}{N_m}\sum_{i=1}^{N_m} (X_i-\mu_m)^2$$
converge to $\sigma^2$, and that the population third absolute moments
$$\kappa_m=\frac{1}{N_m}\sum_{i=1}^{N_m} |X_i-\mu_m|^3$$
are bounded. This could be the assumption, or we could treat the populations as randomly generated and make assumptions about the data generating process that forces these to hold with probability one.
A: No, it does not, since Gaussian distribution converges to Dirac Delta for very small s.d. $\epsilon$, $\delta(x)=\lim\limits_{\epsilon\to 0^+}\frac{e^{-\frac{1}{2}\frac{x^2}{\epsilon^2}}}{\epsilon\sqrt{2\pi}}$.
By CLT, for i.i.d. r.v.s $X_1,\ldots, X_n$, with population mean $\mu$ and finite variance $\sigma^2$,
$\bar{X_n}-\mu \overset{D}{\to} \mathcal{N}(0,\sigma^2/n)$. When we have large sample size  $n$, we have $\epsilon=\dfrac{\sigma}{\sqrt{n}} \to 0$, so the shape of the distribution approaches Dirac Delta, which is a unit impulse function, refer to this: https://en.m.wikipedia.org/wiki/Dirac_delta_function.
For example, you can use the following R code to visualize the shape of the pdf of a standard normal r.v., with variance $\dfrac{\sigma^2}{n}$, for some value of population s.d. $\sigma$ (e.g., 10) different sample size $n$, as shown in the next animation (approaches Dirac Delta with large $n$).
 σ <- 10
 plot(x, dnorm(x, sd = sigma/sqrt(n)), ylab='pdf', type='l', 
         main=paste('normal pdf with variance σ^2/n, σ = 10, n =', n))


A: 
My question is: if the sample size goes closer or even equal to the population size, wouldn't the average values cram so closely near the mean, that violate the "bell shape" distribution?

Indeed, if you sample without repetition from a population, then for $n$ closer to the population size you get a distribution that at some point is exactly equal to the true mean of the population.
But, note that you do not have all conditions that are necessary for the central limit theorem to apply. You need to have the individuals in the sample to be independent. This is not the case when you sample form a finite population without repetition.
A: The central limit theorem applies to infinite sequences of random variables $X_1,X_2,X_3,...$ rather than finite vectors of random variables.  This is evident in the fact that we take the limit $n \rightarrow \infty$ in these theorems.  So what this means is that these problems deal implicitly with an infinite population (or "superpopulation" if you prefer).  When we take this kind of limit, the sample size never goes close to or equal to the population size, since the latter is infinite.  Regardless of how big $n$ gets in the limiting analysis, it is still finite and so it is never close to the population size.
Now, it is of course possible to look at the behaviour of the sample mean for a finite population of size $N \in \mathbb{N}$.  In this case, if you take the sample size $n$ up to the population size $N$ then the sample mean will equal the population mean exactly.  If your analysis conditions on knowledge of the population mean (or treats this as a fixed constant) then the distribution of the sample mean is a point-mass distribution on the population mean value.  Similarly, if the sample mean is close to the (finite) population mean, then its distribution also will not be bell shaped.  (You can think of this like the CLT in reverse; if the number of unsampled datapoints is small then the distribution of the unsampled mean is not well approximated by the normal, and the sample mean is an affine function of this value, so it is also not well approximated by the normal.)
A: Random variables are generally modelled as coming from an infinite population. Even when we're sampling from a finite set of instances, we can still view the population as being infinite. For instance, if you have an urn with 10 red balls and 30 green balls, you can consider your random variable to be "the color of a randomly selected ball", and as long as you do your selection with replacement, you can select balls an infinite number of times. Furthermore, you can model the situation as there being some process that created each of the balls in the urn, and you can imagine what you would get if you were to get an infinite number of balls from whatever process that is.
If you do treat your population size as being finite, and you sample without replacement, then the conditions of CLT are not satisfied. The classical formulation of the CLT requires that the random variables be from identical distributions with a finite variance, and each random variable be independent of each other. The first two conditions can be relaxed for a more general statement, but having the random variables be dependent on each other can reduce or even eliminated the convergence to a gaussian distribution. If you draw from a finite population without replacement, then each draw affects the remaining population, and thus further draws are dependent on earlier ones, violating the conditions of the CLT.
