Using a point estimate in confidence interval calculation

In order to estimate a population parameter(say mean), I read that we use the point estimate and confidence intervals to come up with a range within which the population estimate may lie.

However, my understanding is different.Lets say the point estimate is sample mean here. We can repeatedly keep taking the sample means and then plot all these sample means in a histogram and we would generally observe a normal distribution called the sampling distribution of mean. The mean of this distribution would be a better estimate of the population mean and its standard deviation, called standard error would be sample standard deviation / sqrt(number of points in a sample). Wont the confidence interval(say 95%) range be (sampling distribution mean - 2 * SE,sampling distribution mean + 2 * SE) instead of (point estimate - 2 * SE,point estimate + 2 * SE)?

Why would we use the sample mean(point estimate) in calculating the confidence interval range? What if that particular sample mean was like an outlier in the sampling distribution of mean? In that case, doing +/- 2*SE wouldn't be a good judge to measure population mean right?

The sample mean that you obtain from a single sample of size n can be shown to be the Most Likely Estimate (MLE) of the mean of the sampling distribution of sample means (goodness that is a mouthful!). Of course, if you keep taking samples of size n and averaging all of the sample means you obtain to get a better point estimate of the population mean, the standard error of the mean from all these samples will become small and smaller, and your confidence interval will be tighter and tighter. But the idea is that with just one sample of size n, we can estimate a confidence interval around the point estimate of the population mean (using just the mean from our single sample of size n) within which the true mean of the population will fall, with a specified level of "confidence" or probability. Taking say 5 samples of size n, and using the mean of all five samples, will reduce the standard error by a factor of 5^(1/2). It is effectively the same as taking a single sample of size 5n.

The economics of sampling and our tolerance for an acceptable margin of error determine how big a sample to utilize under different circumstances. If cost and time are no object, the bigger the better!

I suppose you work with the standard (frequentist / classical) methods. In that case "we use the point estimate and confidence intervals to come up with a range within which the population estimate may lie" is not a correct statement. Confidence intervals have to do with the sampling procedure. If you would take many samples and calculate a 95% confidence interval for each sample, you'd find that 95% of those intervals contain the population mean. If you have only 1 sample (thus 1 confidence interval), you can't say how likely it is that the population mean is in it. It is, or it is not.

"We can repeatedly keep taking the sample means....we would generally observe a normal distribution called the sampling distribution of mean. The mean of this distribution would be a better estimate of the population mean"

Of course. More samples = more data = better estimate. Even better would be not taking samples but examining the total population. In reality this is not practical, hence the samples.

*"What if that particular sample mean was like an outlier in the sampling distribution of mean? In that case, doing +/- 2*SE wouldn't be a good judge to measure population mean right?*"

True. Therefore I am convinced that the Bayesian method is better suited for answering real life questions. You start with modeling all the information that you have before you take your samples (this is called the prior), then you take your sample and you calculate the most likely range of the population mean, given your prior and your data.