# Statistical comparison of data obtained from Bootstrapping?

I know that there are many pages about "bootstrapping statistics" and I have read several of them. However, either I am too stupid to understand them, or they do not represent what I am trying to do. Also, some "all-in-one" packages for R are not usable for my case, I think.

Let's say I have some imperfect (e.g., relatively huge variance), biological data of certain traits of an animal in response to a food/nutrient gradient. My goal is to estimate food/nutrient thresholds where no more increase in this trait can be observed. There is a method of Sperfeld and Wacker (2011), where they fitted a Monod function to their gradient data and estimated µmax (asymptotic maximum), Ks (half saturation), and the desired threshold (at a certain level of increase, e.g., 75 %; as µmax is not meaningful in a biological sense). Since we would also like to obtain the uncertainty of these values, they performed bootstrapping of their data, from which they estimated the "real" threshold and its uncertainty from the original data.

Here is a picture from that publication that may help to understand the approach:

So far, so easy...

Now say, I have not just one nutrient, but several nutrients that affect each other. Therefore, I am estimating the thresholds etc. at different conditions (concentrations of a second nutrient), resulting in a new "data set" of thresholds (and their variance) under these conditions, which follow again a kind of saturation curve (or non-linear function).

I would like to model this relation of the estimated thresholds and the second nutrient, by using, e.g., a GAM, and to estimate at which point of the curve (or which concentration), there is no significant change any more. To do that, I could use an approach of Gavin Simpson of a smooth trend analysis.

However, here comes the actual question and thoughts:

Would this "simple" approach, using a GAM, be valid?

I've read a lot of things, that I cannot use simple statistics on bootstrapped values. However, I derive a "totally new" data set, since I did not just bootstrap the original data and analyse them, but rather use bootrapping to derive "new" or other data, which I then want to compare. More or less simple methods mentioned in the beginning, usually do not include such a case like mine, why I am quite uncertain how to apply this idea.

If the answer of the question is 'NO!', what would be a valid approach, or how I would have to modify it?

I am no mathematician or statistician. Even though I would appreciate any help, I would prefer an (additional) answer in more or less simple words ;)

Thanks a lot!

I derive a "totally new" data set, since I did not just bootstrap the original data and analyse them, but rather use bootrapping to derive "new" or other data, which I then want to compare.

It seems that you are using the term " 'totally new' data set" to describe the estimates of thresholds as functions of other nutrients from your models. If so, that's probably not the best way to describe them.

Think about them instead as predictions from your model, even if the predictions are smoothed relationships between the threshold for one nutrient and the values of another. Then what you have is simply a model estimate/prediction, which you can try to evaluate via bootstrapping. It's a continuous relationship instead of a single point estimate.

It helps to keep the bootstrap-resampling issues separate from those of the modeling. Bootstrapping is simply resampling with replacement from the original data. The idea is that the process of bootstrapping mimics the sampling of the original data set from the underlying population.

A way to apply bootstrapping here is to fit your multi-variable model on the original data first. Estimate the set of nutrient thresholds of interest as functions of other nutrients from that model, smoothing however you will. If you are going to evaluate the modeling process by bootstrapping, I don't see a need to estimate the variance in those estimates at this stage.

Then repeat the entire modeling process on multiple bootstrapped samples of the data. You will get a set of smoothed curves of thresholds versus nutrient concentrations for each model from a bootstrapped sample.

The variability of those curves among those models is what you want to display. That's similar to your example from the literature, which displays several estimated growth versus concentration curves. You would show several estimated threshold versus (other) nutrient concentration curves. How best to summarize the variability among those curves would depend on the nature of the curves and your understanding of the subject matter. For example, if there is an accepted general functional form for such threshold-versus-nutrient curves in your field, your smoothing might best be done with respect to that form and you could report the variance among coefficient estimates from the multiple models.

Note that bootstrapping won't necessarily work well here. First, estimates of things like thresholds can be quite noisy. Second, insofar as threshold estimates rely on extremes of the set of original observations, bootstrapping might be extremely unreliable. Third, it can't overcome problems of having a limited or unrepresentative data sample to start with. See this page and its links for cautions. It typically works OK for better-behaved things like regression coefficients and associated predictions well within the range of observations.

• Thanks for your elaboration! I'll try to put it in other words, to be sure, that I understood you correctly: I should apply a curve or model to my original data (at best including the relationship between both nutrients), estimate thresholds from that model of one nutrient under different values of the second nutrient (i.e., generating curves of this relation), and then repeat the procedure with bootstrapped data. Then I receive a bunch of values/curves that I could model/analyse according to typical approaches of my field. Did I get it correctly?
– Maki
Sep 8, 2022 at 9:02
• Thanks also for the link. I haven't have a look into the books, but I definitely will!
– Maki
Sep 8, 2022 at 9:02
• @Maki what you describe is what I recommend. If there are other established approaches in your field, however, do consider using them. Sometimes there are subtleties arising from subject-matter-specific issues that make generic suggestions like mine less useful.
– EdM
Sep 8, 2022 at 15:15
• thanks for your advice, and also the awareness for caution. I think the data are quite representative, as far as experimental data can be. As far as I know, there is no common way for that, since no one did something like this that way. However, actually this is exactly what I already tried (even though I was a bit uncertain), but I didn't want to bias the suggestions or discussion in the direction of my idea, rather keeping the possibility of getting other ideas/solutions. I'd be curious about other opinions/ideas, otherwise I'll mark yours as solution in a day or so... Thanks again!
– Maki
Sep 9, 2022 at 5:57