Difference between Sampling a population Vs Bootstrapping I am finding it difficult to understand the concept of Bootstrapping in statistics . I know what sampling is , that is , taking a 'sample_size' number of observations from a population to estimate some of that population statistics like mean , SD etc . I thought bootstrapping was doing that same  process of sampling multiple times , but it doesn't look like that's a proper way to put it . Some sources say bootstrapping takes a number of samples with size equal to the original dataset while some others say it takes samples of desired sample size from within a  bigger sample of a dataset. All of these definitions got me confused .
Could someone please explain the difference between the two in simple and intuitive manner ? i.e , what exactly is each one of them doing ?
 A: When you take a sample from a population, you are gathering information about the population, which you might use for making a confidence interval or for testing a hypothesis about population parameters (perhaps the population mean $\mu).$
When you 're-sample' as in bootstrapping, you are analyzing data already taken from a population. Re-sampling does not provide any new information about the population. (But it
might help you better to understand the data you already have.)
You refer to two kinds of 're-sampling' that are in common use.

*

*In making a nonparametric bootstrap confidence interval, based on $n$ existing observations, you might take a large number $B$ of re-samples from your data. You would sample with replacement and re-samples would be of size $n.$
Nonparametric bootstrapping is often used when you do not
know the distribution 'family' of the population. (In particular, you would rarely use a nonparametric bootstrap for data known to have been sampled from a normal or an exponential population.)

*In making a parametric bootstrap, you typically know the distribution family of the population, but perhaps not the values of particular parameters (such as $\mu$ or $\sigma.)$.
In that case, you use the data to estimate parameters(s) and then use bootstrapping to get a confidence interval to go with each estimate. Then you use the estimated parameter value(s) to simulate a re-sample of size $n$ from the population. So you are not re-sampling directly from the data but from a population suggested by the data.
Nonparametric bootstrap CI for population mean.
Suppose I have a vector y that contains $n$ observations from a population of unknown distribution. I want to make a 95% nonparametric bootstrap confidence interval for the population mean $\mu.$
Here are summary statistics and a histogram:
summary(y); sd(y)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   5.83   11.85   16.16   16.64   19.27   53.99 
[1] 7.774901   # sample SD
hist(y, prob=T, col="skyblue2");  rug(y)


Here is R code to make a simple quantile bootsrap CI for $\mu.$ I take a large number $B = 3000$ re-samples of size
$n = 50$ from y and find the average $\bar X^*$ of each re-sample [denoted as a.re in the R code.].
set.seed(2020)
a.re = replicate(3000, mean(sample(y, 50, rep=T)))
quantile(a.re, c(.025, .975))
    2.5%    97.5% 
14.65229 18.95220 

So I could say that a 95% nonparametric bootstrap CI
for $\mu$ is $(14.7, 19.0).$ The interval contains the
sample mean $\bar Y = 16.64,$ but $\bar Y$ is not necessarily at the center of the CI. [Confession: Because I can see
form the histogram that the sample is skewed, it might
have been better to do a bias-corrected CI, but I'm
trying to illustrate re-sampling, but get into
nuances of various types of bootstrap CIs.]
Below is a histogram of the re-sampled averages a with
vertical lines indicating the bootstrap CI for $\mu.$
hist(a.re, prob=T, col="wheat")
abline(v=q, col="blue")


Parametric bootstrap CI. Now suppose I know that
that the sample y came from a gamma distribution with shape parameter $\alpha = 5$ and unknown rate parameter $\lambda.$ A reasonable estimator of $\lambda$ is
$\hat \lambda = \alpha/\bar Y = 5/16.64 =  0.30.$
Now I simulate a large number $B = 3000$ samples of size
$n = 50$ from the distribution $\mathsf{Gamma}(\alpha=5, \lambda = .3).$ Then I find $\hat\lambda^*$ from each sample.  At the end, I can use $(0.27, 0.34)$ as a 95% parametric bootstrap CI for $\lambda.$
set.seed(2020)
lam.re = replicate(3000, 5/mean(rgamma(50, 5, .3)))  
q = quantile(lam.re, c(.025,.975));  q
     2.5%     97.5% 
0.2668468 0.3416872 

Here is a histogram of the bootstrap distribution of lam.re along with vertical bars showing the 95% parametric bootstrap CI for $\lambda.$ [There are ways
to use the gamma distribution to make a CI for $\lambda$
without bootstrapping. But in this case the bootstrap method works very well.]
hist(lam.re, prob=T, col="wheat")
abline(v = q, col="blue")


Remark. For both of these bootstrap CIs I did
re-sampling. But in these procedures I sample no
additional data from the population.
Note: In this case the 'population' was R's
function to generate gamma data. Data y for this demo
were sampled as follows;
set.seed(2020)
y = round(rgamma(50, 5, .3), 2)

