I am having trouble learning the relevant concepts of t-test. Even if you search the web or books, the explanation is slightly different, it causes confusion. (Please understand that I am not a statistics or math major)
Many of the sites and books you're likely to be looking at will contain errors, or at best infelicities of explanation which will lead you to be confused; in many cases it's quite clear that the authors of lots of web pages and even a great many textbooks are themselves no less confused than you are (for all that they appear to have convinced themselves otherwise. You have the distinct advantage over such sources of not having convinced yourself that you understood what you did not.)
I think the first thing to do is to lay out two quite distinct concepts, which too often get conflated:
What is specifically assumed in order to (algebraically) obtain a t-distribution for the null distribution of the test statistic when $H_0$ is true, and hence for obtaining the exact significance level you seek, and thereby, 'exact' p-values. These assumptions to derive that null distribution of the test statistic are usually called - for obvious reasons - "the assumptions" of the test. If you break those assumptions, you will not get an exact t-distribution for the distribution of that test statistic when $H_0$ is true.
The weaker conditions under which the t-statistic will have a null distribution which is reasonably approximated by a t-distribution.
There's a difficulty here that should be signalled, in that what's 'reasonable' is very much a case-by-case thing. What you might accept as reasonable for one task and what I might accept as reasonable for that task will differ, and they will probably differ again when we change to some other circumstance.
This distinction is very important for understanding many of the differing claims you'll encounter, and for figuring out whether they make sense as is, whether they make sense in some circumstances (such as adding some conditions) or whether they're pretty much just wrong.
Someone say that the population should normally distributed.
This is situation 1 above -- the assumptions under which the t-distribution is derived for the null distribution of the t-test statistic is that the set of random variables representing the sample we plan to draw are independent, identically distributed, normal.
Someone say that the distribution of sampling mean(for multiple trial of sampling) should be normally distributed.
You don't need to run the sampling 'many times'; this is a misconception that many books create. The sample mean of a single sample is a random variable. We can talk about the distribution of that one random variable, and that is indeed the sampling distribution of the mean.
Strictly, with i.i.d. observations and finite sample size, you can only get a normal distribution for the mean if you started with a normal distribution for each of the values being averaged. The best we can hope for is that the sample mean is approximately normally distributed. We are here talking about whether we're dealing with a sampling distribution that's close enough to use a t-approximation now -- i.e. under situation 2; clearly we aren't in situation 1.
Even if you have that, note that the t-statistic has both a numerator and a denominator; you can't ignore what happens to the denominator, and you must consider that the statistic is a ratio of the two. The finite-sample distribution of the test statistic depends on the behavior of the ratio of the numerator to the denominator, not just on the numerator.
To deal with the numerator, people try to invoke the CLT and declare victory, but as we see, it's not quite that simple. We have multiple things to consider there.
One thing to beware: many books are quite misleading about what the CLT actually says. Indeed many are flat out wrong. Fortunately, some of those books do convey a somewhat different fact -- that in many cases, the distribution of sample means may sometimes be fairly well approximated by a normal distribution (while never actually attaining it). As suggested above, that does not of itself make the t-statistic have a t-distribution.
Nevertheless, it's often the case that when the numerator is close to normal, the whole statistic is also pretty close to a t-distribution. This is more of an empirical observation than a proven fact, although last I saw, it did seem like we're perhaps getting quite close to having some theorems that will give some conditions under which we can say more about when the sampling distribution of the t-statistic will be within a certain distance (in a specific but useful sense) of the of the t-distribution.
Skewness and heavy-tailedness are problematic; in particular it's easy (particularly in the case of a one-sided one sample test) for the significance level to be inflated above the chosen significance level with substantial skewness. I've seen a number of real data sets where the distribution was so skewed that samples in the hundreds or even thousands might be required to get good approximate significance levels (though we might easily have different opinions about what's 'good enough' for some situation), and that problem can be greatly increased if you're adjusting for multiple testing (because as you go further into the tail, skewness remains a problem for longer). Heavy tailedness will mostly tend to push down significance levels. This may not seem to be a big problem, but if you think about power it's compounding the power-loss you're already suffering for using a suboptimal statistic by also conducting the test at a lower significance level than you wanted. Sometimes quite considerably lower.
On the other hand, someone say that the each value of samples shoud be normally distributed.
This is again, situation 1. In fact I can't see any difference here from the previous thing you said which was situation 1.
in some cases the normality test was performed on each sample value prior to performing the t-test, and in other cases, the normality test was performed on the sampling mean (average of samples).
I would not suggest performing a normality test at all. It's answering the wrong question, it leads people to change their hypotheses based on what they discover in the data, and it sometimes leads to some distinctly counter-productive actions.
(That's not to say that I think the assumptions should be ignored, just that this isn't necessarily a great way to go about considering them.)
(1) I think the normal distribution of sampling mean is required to evaluate the possibility of certain t-value appears. (Here, I think that the mean value could explain t-statistic)
Please see the explanation above.
(2) If the statement:"the normal distribution of sampling mean is required" is true, isn't it always normal distributed as CLT says?
The CLT says nothing like that.
The relevant version of the CLT says (more or less) that in the limit as $n$ goes to infinity, the distribution of the standardized sample mean $(Ȳ−μ)/(σ/√n)$ goes to a standard normal distribution. It does not say that sample means actually have normal distributions at any finite sample size (and in practice this is not so unless you already had normal distributions to begin with).
We're instead dealing with how close to normal a sample mean might be and (again) we should keep in mind that this is not sufficient on its own to establish how good the t-distribution is as an approximation to the null distribution of the test statistic, because a t-statistic is not simply a mean; its denominator is a random quantity, and when you don't start with normal variates, one that is dependent on the numerator.
Since the distribution of the sampling mean is an imaginary distribution(In practice, we only do one set of sampling),
Again, you're confused about what's going on. This confusion is caused by books trying very hard to avoid talking about random variables, but then proceeding to assert facts that are really about random variables. Good luck trying to make heads or tails of the result of that!
I'm going to try to at least get you partway (though it's not formally correct, I'll at least try to convey some kind of understanding the distinctions here).
Let's think about the distribution that comes from rolling a fair die. This is a discrete uniform distribution, with probability $\frac16$ on each of the values $1, 2, ..., 6$.
A single die roll has this distribution! Of course, once you observe the outcome of the roll - once we come to consider the observed outcome - we're not discussing a random quantity any more but a realization of it, and it just has a single number as its value; probability doesn't enter into that realized value, probability describes the behavior of the random quantity, rather than the nonrandom observation.
The mean of a single sample is similar; it has a distribution, and we can derive that distribution -- the sampling distribution of the mean. Of course, once you observe the sample, the observed sample mean is the realization of that random value. That observed mean is not itself random, it's a fixed value.
Then we are conducting our t-test, which evaluates the possibility of our sample t-statistic within that imaginary distribution of t-values. Is this wrong statement?
The t-statistic is also a random quantity. If the sample values are i.i.d. normal and $H_0$ is true, then the t-statistic has a t-distribution (again, we can talk about the sampling distribution of a t-statistic under some set of conditions). Of course, once you observe the sample, you have a realization of the value of that random quantity, it's just a number.
There's no need to observe many samples for any of this discussion, since we're really talking about the distribution for one sample statistic. One reason you might do that, though, is simply to get an approximate look at what the sampling distribution looks like in a case where you can't readily derive its exact form, by relying on the approach of a sample distribution (e.g. the sample cdf) to the population distribution (population cdf). That is, we might illustrate the approximate shape of the sampling distribution of a single t-statistic, by sampling many of them (and relying on something like the Glivenko-Cantelli theorem, which tells us that the sample cdf of some random quantity converges to the population cdf from which we were obtaining samples of it).
Which is to say, even though any one observed outcome is just a number, nevertheless, the empirical cdf of a large collection of (i.i.d.) samples from the same distribution will approach the population cdf. So such sampling does help us see (in a somewhat rough way) what the distribution of the population looks like that we're sampling, even though we're talking about what the distribution of a single random value would be.
By way of illustration, I don't know what the distribution of a t-statistic will be when sampling from a lognormal distribution. It won't be algebraically tractable and I don't even think its readily doable numerically (though the convolution for the distribution of the numerator would be fairly straightforward). But we can simulate it to get a good idea of what the distribution is like. Here's an example with shape parameter $\sigma=1$, and samples of size $60$. Here I draw a Q-Q plot:
Which shows us that the one-sample t-statistic is pretty left skew; a histogram shows the same thing:
This left skewness might surprise many people, because the numerator of the t-statistic is distinctly right skew. You really can't forget about that denominator in small samples -- it matters!
Now even though we simulated many samples (100,000 samples each of size 60), we should not be confused by this into thinking that we're doing this to find the distribution for many samples; we're doing it to get an approximate picture of the sampling distribution of the t-statistic calculated on a single sample of size 60. That histogram is a reasonably accurate picture of what the density will look like.
[Note that the parent distribution in this case is distinctly less skew than a number of real-life distributions I have dealt with. The ordinary moment-skewness is less than 6.2. That's not small, but considerably bigger skewness than that certainly occurs in practice - and in situations where you might still be interested in inference about means.]
(3) I am dealing with one-sample t-test. Is there any requirements differences between one-sample and other t-tests?
Well, yes, in several senses. In situation 1, you have to have more things be simultaneously true for the t-statistic to exactly have a t-distribution when $H_0$ is true (both samples drawn from normal 'populations', independence both within and between samples, equal variances). In situation 2, you're somewhat less sensitive to skewness under $H_0$ when the two populations have the same shape (you can still lower the type I error rate, just as in the one-sample case with a heavier tail), but it's harder to push it up very much above the type I error rate (particularly around $\alpha=0.05$)
If there's anything you'd like clarification on, please indicate where it's required.
Please note that there are a number of relevant posts already on site which will shed more light on your questions and related questions that might arise from the discussion here.
