I got a comment on a paper that I recently submitted.
He said, "Pag 7: referring to the “Univariate Analysis” section, bootstrap is not mentioned. This technique is extremely useful when dealing with datasets having few observations compared to the number of features, as often happens for metabolomic data. Is bootstrap performed by Metabox? If not, this should be highlighted since it is a limitation."
At page 7, I wrote, "• Univariate Analysis. Metabox collects a variety of well-established statistical hypothesis testing methods and post hoc analysis procedures (Table 1). In addition to the hypothesis testing procedures provided, metabox includes corresponding non-parametric testing procedures, post hoc analysis with false discovery rate (FDR) correction on both main effect level and simple main effect level, and power analyses at entity-level. Furthermore, metabox automatically and appropriately suggests statistical analysis methods according to the user-input study design. This feature aims to aid users through the depths of statistical terminology."
In the univariate analysis part, it is just a basic hypothesis testing procedure, including t-test and ANOVA, etc.
My question is that how to implement bootstrapping to, for example, t-test?
To my understanding, the bootstrap is just a resampling with replacement, which can be used to estimate the variation of a statistic. But when we do a hypothesis testing, does it make sense to estimate the variation of, for example, t (or p-value)?
Or do I misunderstand the comment?
Thanks a lot.