In order to estimate a population parameter(say mean), I read that we use the point estimate and confidence intervals to come up with a range within which the population estimate may lie.
However, my understanding is different.Lets say the point estimate is sample mean here. We can repeatedly keep taking the sample means and then plot all these sample means in a histogram and we would generally observe a normal distribution called the sampling distribution of mean. The mean of this distribution would be a better estimate of the population mean and its standard deviation, called standard error would be sample standard deviation / sqrt(number of points in a sample). Wont the confidence interval(say 95%) range be (sampling distribution mean - 2 * SE,sampling distribution mean + 2 * SE) instead of (point estimate - 2 * SE,point estimate + 2 * SE)?
Why would we use the sample mean(point estimate) in calculating the confidence interval range? What if that particular sample mean was like an outlier in the sampling distribution of mean? In that case, doing +/- 2*SE wouldn't be a good judge to measure population mean right?