I'm trying to perform variable selection methods on a small data set (approximately 100 observations with 14 predictors each). Is there any way to increase the sample size with bootstrapping? Are there any methods for bootstrapping multivariate data? If so, what are the concerns about bootstrapping multivariate data to increase sample size and how can I generate this data in R or Python?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
- Is there any way to increase the sample size with bootstrapping? No. The bootstrap is basically a technique to approximate standard errors of complex estimators. It is not a data generating technique.
- Are there any methods for bootstrapping multivariate data? Yes. Usually, the data lines are resampled together as a whole (see code below). This strategy may fail e.g. if the order of the lines is relevant to the problem (like in time series analysis).
How to e.g. bootstrap lines of a multivariate data set $X$:
getBootSample <- function(X) {
X[sample(1:nrow(X), replace = TRUE),]
}
set.seed(5)
head(getBootSample(iris))
-
1$\begingroup$ This is a nice answer. I don't know where the notion that "bootstraps increase sample sizes" ever came from... Perhaps that the exact tests from bootstrap can have asymptotic-like properties in small samples, or that (as a consequence) you wind up with seemingly more rows of data. More efficient analyses could be afforded using a parametric bootstrap, but that's a silly notion since that's far too assumption dependent. $\endgroup$– AdamOCommented Aug 6, 2015 at 18:23
-
$\begingroup$ As it often makes sense to model many bootstraps and aggregate, the same samples are drawn alot of times. $\endgroup$ Commented Aug 6, 2015 at 18:42