When is the distribution of $(\overline{x}-\mu)/{\rm SE}(\overline{x})$ normal and when is it $t$? When estimating the mean of a population, two distributions seem to be useful, the normal distribution and the t-distribution.
Is it correct to say that $t=(\overline{x}-\mu)/{\rm SE}(\overline{x})$ follows a normal distribution for ANY population (not just normally distributed), as long as the samples sizes are significant in size (by means of the central limit theorem) ?
And, is it correct that t follows a t-distribution when the sample size is small, but then the population only if the population is normally distributed, because the central limit theorem does not apply ?
Thanks !
 A: There's a subtle issue here that is not mentioned in the question regarding estimation of the standard deviation of $\overline{x}$'s sampling distribution. 
Suppose you have a sample from a population with mean $\mu$ and variance $\sigma^2$. When $\sigma^2$ is known, $${\rm SE}(\overline{x}) = \sigma/\sqrt{n}$$ is exactly the standard deviation of the sample mean. In practice, you usually don't know $\sigma^2$, so you instead plug in the sample variance $\hat\sigma$ to use $${\rm SE}(\overline{x}) = \hat\sigma/\sqrt{n}$$ to estimate the standard deviation of $\overline{x}$. This distinction is actually important - when the variance is unknown, this additional uncertainty must be incorporated into the hypothesis test. This is why, even when the sample is normally distributed, the test statistic has a $t$-distribution (which has longer tails) instead of a normal distribution when $\sigma$ is unknown. 
Is it correct to say that $t=(\overline{x}−μ)/SE(\overline{x})$ follows a normal distribution for ANY population (not just normally distributed), as long as the samples sizes are significant in size (by means of the central limit theorem)
This is almost correct. The population must have finite variance (i.e. not have tails that are "too long") for this to be the case. Even when the population does have a finite variance, the population distribution can have a large effect on how long until the CLT "kicks in". For shorter tailed distributions this convergence is faster. For long-tailed distributions it can take quite a while (e.g. see my example here). 
Note that since we're talking about a "large sample" result here, this is true regardless of whether or not you know $\sigma$ since $\hat \sigma$ gets closer to the true $\sigma$ as the sample size increases. 
And, is it correct that t follows a t-distribution when the sample size is small, but then the population only if the population is normally distributed, because the central limit theorem does not apply? 
Again, assuming we're in the "$\sigma$ is unknown" world, $t$ only follows a $t$-distribution when the sample is normally distributed, which I think is what you're saying here. Related to what I said in the beginning, if $\sigma$ is known, then $t$ will have a (exact) normal distribution if the sample is normally distributed. 
To summarize: 


*

*If $\sigma$ is known, and the population is normally distributed: $t$ has a normal distribution. 

*If $\sigma$ is unknown, and the population is normally distributed: $t$ has a $t$-distribution. 

*If the population is not normally distributed but meets the regularity requirements of the CLT: $t$ has an approximate normal distribution whether or not $\sigma$ is known. That is, the distribution of $t$ converges to a normal distribution as the sample size increases. 
A: No the standard error of the mean would be related to the chi square distribution not the normal when sampling froma normal distribution.  It is the sample mean normalized by the the samplestandard error that is approximately normal by the CLT and has a t distribution when the true mean is zero and the data are normally distributed. 
