This is a follow-up on the terminology used in this answer. In brief, I was surprised by the term bootstrapping being used more broadly than what I have mostly encountered so far.
My Understanding of the Terminology so far
The following is is based on several articles and other material on bootstrapping. Admittedly they were all rather vague, so I am attempting a definition below. A statistics professor confirmed the main distinction I am making here.
Bootstrapping
Given a real dataset $X$ sampled from some population $P$, a bootstrap is an artificial dataset that aims to represent another sample from the population $P$, but is constructed using only knowledge of $X$. Bootstrapping is any technique exploiting the statistical properties of bootstraps.
For example if we have a dataset $X$ of paired numbers, we can generate a bootstrap by resampling this dataset, leaving the pairs intact. Let $r$ being a correlation coefficient and $\hat{R}$ the distribution of $r$ of an appropriate amount of such bootstraps. We can then estimate the confidence interval of $r(X)$ by looking at the width of $\hat{R}$. However, we would expect $r(X)$ to be somewhat central within $\hat{R}$ (i.e., not an outlier).
Not Bootstrapping
I have seen this being called Monte Carlo null model, just null model, or surrogate. I here use the latter, because it is more compact, but I acknowledge that it is not widely used.
A surrogate is a dataset that aims to represent a null model but inherits some properties from the original dataset $X$. We can build a hypothesis test based on this by comparing a statistic for $X$ and the surrogates, where we reject the corresponding null hypothesis when the statistic for our original dataset is more extreme than for the surrogates.
In the correlation example, suppose our null hypothesis is that the data is uncorrelated. We can produce a corresponding surrogate by repairing our dataset (thus destroying the pairs). Let $\tilde{R}$ be the distribution of $r$ of an appropriate amount of such surrogates. If our null hypothesis is false, we expect $r(X)$ to be extreme in comparison to $\tilde{R}$, i.e., be an outlier. Otherwise, we expect $r(X)$ to lie within $\tilde{R}$.
The Simulations in Question
My understanding of the simulation part of this answer by W. Huber is this: Our null model is that the data is generated by an inhomogeneous Poisson process. We estimate the event rate of this process from the original data via a GLM. Then we simulate 2000 artificial datasets complying with the null model, i.e., instances of the inhomogeneous Poisson process. As the dispersion for the original data is much lower than for all artificial datasets, we can reject the null hypothesis: The dispersion of the original data is significantly low.
I was surprised by the usage of bootstrapping for this: The artificial datasets here do not aim to reproduce whatever population generated the original data. We do not try to estimate an interval of confidence for the dispersion of the underlying population or similar. I would call the artificial datasets surrogates. I remarked this to which W. Huber replied:
Yes, it is honest-to-God bootstrapping. There are various flavors. This one is parametric in the sense of assuming the data arise as independent realizations of Poisson variables--in effect, an inhomogeneous Poisson process. There is no "null model" or other hypothesis in effect.
Actual Question
- Is there any somewhat authoritative resource that confirms or refutes my definitions given above?
- In case of confirmation, did I somehow miscategorise W. Huber’s approach?
- In case of refusal, is there any terminology to distinguish the two kinds of artificial datasets (what I call bootstraps and surrogates above).
- Either way, how is there not a null model and hypothesis as I identify above?
