Statistical comparison of data obtained from Bootstrapping?

Question

I know that there are many pages about "bootstrapping statistics" and I have read several of them. However, either I am too stupid to understand them, or they do not represent what I am trying to do. Also, some "all-in-one" packages for R are not usable for my case, I think.

Let's say I have some imperfect (e.g., relatively huge variance), biological data of certain traits of an animal in response to a food/nutrient gradient. My goal is to estimate food/nutrient thresholds where no more increase in this trait can be observed. There is a method of Sperfeld and Wacker (2011), where they fitted a Monod function to their gradient data and estimated µmax (asymptotic maximum), Ks (half saturation), and the desired threshold (at a certain level of increase, e.g., 75 %; as µmax is not meaningful in a biological sense). Since we would also like to obtain the uncertainty of these values, they performed bootstrapping of their data, from which they estimated the "real" threshold and its uncertainty from the original data.

Here is a picture from that publication that may help to understand the approach:

So far, so easy...

Now say, I have not just one nutrient, but several nutrients that affect each other. Therefore, I am estimating the thresholds etc. at different conditions (concentrations of a second nutrient), resulting in a new "data set" of thresholds (and their variance) under these conditions, which follow again a kind of saturation curve (or non-linear function).

I would like to model this relation of the estimated thresholds and the second nutrient, by using, e.g., a GAM, and to estimate at which point of the curve (or which concentration), there is no significant change any more. To do that, I could use an approach of Gavin Simpson of a smooth trend analysis.

However, here comes the actual question and thoughts:

Would this "simple" approach, using a GAM, be valid?

I've read a lot of things, that I cannot use simple statistics on bootstrapped values. However, I derive a "totally new" data set, since I did not just bootstrap the original data and analyse them, but rather use bootrapping to derive "new" or other data, which I then want to compare. More or less simple methods mentioned in the beginning, usually do not include such a case like mine, why I am quite uncertain how to apply this idea.

If the answer of the question is 'NO!', what would be a valid approach, or how I would have to modify it?

I am no mathematician or statistician. Even though I would appreciate any help, I would prefer an (additional) answer in more or less simple words ;)

Thanks a lot!

Answer 1

I derive a "totally new" data set, since I did not just bootstrap the original data and analyse them, but rather use bootrapping to derive "new" or other data, which I then want to compare.

It seems that you are using the term " 'totally new' data set" to describe the estimates of thresholds as functions of other nutrients from your models. If so, that's probably not the best way to describe them.

Think about them instead as predictions from your model, even if the predictions are smoothed relationships between the threshold for one nutrient and the values of another. Then what you have is simply a model estimate/prediction, which you can try to evaluate via bootstrapping. It's a continuous relationship instead of a single point estimate.

It helps to keep the bootstrap-resampling issues separate from those of the modeling. Bootstrapping is simply resampling with replacement from the original data. The idea is that the process of bootstrapping mimics the sampling of the original data set from the underlying population.

A way to apply bootstrapping here is to fit your multi-variable model on the original data first. Estimate the set of nutrient thresholds of interest as functions of other nutrients from that model, smoothing however you will. If you are going to evaluate the modeling process by bootstrapping, I don't see a need to estimate the variance in those estimates at this stage.

Then repeat the entire modeling process on multiple bootstrapped samples of the data. You will get a set of smoothed curves of thresholds versus nutrient concentrations for each model from a bootstrapped sample.

The variability of those curves among those models is what you want to display. That's similar to your example from the literature, which displays several estimated growth versus concentration curves. You would show several estimated threshold versus (other) nutrient concentration curves. How best to summarize the variability among those curves would depend on the nature of the curves and your understanding of the subject matter. For example, if there is an accepted general functional form for such threshold-versus-nutrient curves in your field, your smoothing might best be done with respect to that form and you could report the variance among coefficient estimates from the multiple models.

Note that bootstrapping won't necessarily work well here. First, estimates of things like thresholds can be quite noisy. Second, insofar as threshold estimates rely on extremes of the set of original observations, bootstrapping might be extremely unreliable. Third, it can't overcome problems of having a limited or unrepresentative data sample to start with. See this page and its links for cautions. It typically works OK for better-behaved things like regression coefficients and associated predictions well within the range of observations.