importance of CLT in t-test and z-test I was going through the assumptions of z-test and t-test, all most all of the references mention that the data should be normally distributed. There is no mention of estimator's distribution. If the distribution of the data is what we care about, why do we talk about CLT which only applies to the estimator? So this raises to the question.


*

*Does estimator being normally distributed help us in choosing t or z test even if the data is not normally distributed?

 A: Data drawn from a normal distribution assures that the z-score has a normal distribution. The central limit theorem only says that the z-score converges to normality, and it doesn’t even say how fast, so our 30 samples might not result in a very normal-like z-score (though the convergence is often quite fast...while it’s just a joke, there’s a reason that I say that statisticians think $30=\infty$).
A: @Dave mentions the speed of convergence in the Central Limit Theorem.
The shape of the distribution from which samples are taken can make
a big difference in the speed of convergence.


*

*Means of samples of size 15 from a uniform distribution are very nearly
normally distributed. So a sample of size $n = 30$ would easily be large enough to use t tests.

*Means of samples of size 50 from an exponential distribution are not normal. So a sample of size $n = 30$ would not be large enough
to use t tests (150 would be a lot better).


Usually, the 'rule of 30' is accompanied by warnings that it may not
apply when data show extreme skewness or many outliers. Figure below
shows two histograms, each for 10,000 means. 
Means of 15 uniform
observations in the left panel, and means of 50 exponential observations
at right. Red curves are normal curves that match the mean and SD in
the respective histograms. (R code for the simulations and making the figure is shown below the figure.)

set.seed(2020)
a.unif = replicate(10^4, mean(runif(15, 0,30)))
a.exp = replicate(10^4, mean(rexp(50, 1/15)))

par(mfrow=c(1,2))
 hist(a.unif, prob=T, col="skyblue2", 
      main="n=15: Sample Averages of Uniform Data")
  curve(dnorm(x, mean(a.unif), sd(a.unif)), add=T, col="red", lwd=2) 
 hist(a.exp, prob=T, col="skyblue2", 
      main="n=50: Sample Averages of Exponential Data")
  curve(dnorm(x, mean(a.exp), sd(a.exp)), add=T, col="red", lwd=2)
par(mfrow=c(1,1)) 

About outliers: Samples of size 30 from a uniform distribution very seldom have boxplot outliers--less than 1 in 100 do. By contrast, about 75% of samples of size 30 from
an exponential distribution have outliers in a boxplot and most have their upper whisker longer than the lower one, indicating upward skewness. So data frequently show warning signs
when t tests are inappropriate.
All
histograms in the figure below are based on samples of size 30,
from uniform distributions at the top, exponential distributions in the bottom panel.

A: The other answers here are already very good. I'll add a very short additional one.

I was going through the assumptions of z-test and t-test, all most all of the references mention that the data should be normally distributed.

If your data comes from a normal distribution, then the mean is normally distributed.
If your data comes from a (possibly non-normal) distribution, then your mean is asymptotically normally distributed by the CLT.
Normality of the data is sufficient but not necessary for the mean to be normal "enough".
Also: there is unfortunately an enormous amount of misinformation out there, which explains the references you have been seeing.

If the distribution of the data is what we care about, why do we talk about CLT which only applies to the estimator?

We usually care about the distribution of the estimator in inferential statistics, since we are comparing a test statistic to an asymptotic distribution. The distribution of the data are only of secondary interest. And see above on the misinformation out there.
