I have a dataset that is structured as follows:

  • 6 Subjects were observed
  • 2 measurements (continuous values X and Y) were taken for each subject at 7 different time points (baseline and 6 additional times)
  • At baseline, all the measurements for X for all subjects is the same (actually 0), while the Y measurement varies.
  • The measurements, X and Y, are know to be related.
  • If we take all the data and plot X vs Y, we see a nonlinear relationship between the two measurements (see hand drawn, example plot below).

The interest is determining at what point, X, can we say that Y is significantly different from baseline? Using the baseline measurements, we can calculate a confidence interval to establish something to test against.

Seems an easy enough question to answer, but there are some complexities that limit the approaches I've take thus far. Here's what I've done and the limitations of each approach:

  • Find the the intersection of the fit line to the upper 95% confidence interval of the baseline. This is simple to do, but this ignores the uncertainty in the fit and that this is a repeated measures design.
  • Calculate a confidence or prediction interval for the fit line. This takes into account uncertainty in the fit, but ignores the repeated measures part.
  • Use a mixed effects approach to the fit. This takes into account the repeated measures part, but now I can't calculate confidence intervals to assess uncertainty of the fit overall as I have 6 independent fitted curves, one for each subject.

What approach am I missing that allows to account for both the uncertainty of the fit and the repeated measures design?

Hand drawn example

  • $\begingroup$ As a first cut at the problem, how about using the intersection point for the two dotted lines on your graph? $\endgroup$ Mar 3 '18 at 14:08
  • $\begingroup$ Thanks for the comment. Yes, that's an option, but it doesn't account for uncertainty in the fit of the fact that these are repeated measures, correct? As I mentioned, I can compute confidence or prediction intervals of the fit and look for a similar intersection, but again, the repeated measures question remains. $\endgroup$
    – KirkD_CO
    Mar 3 '18 at 14:41
  • $\begingroup$ I just modified my question to describe what approaches I've already tried and the limitations of each. $\endgroup$
    – KirkD_CO
    Mar 3 '18 at 15:00

This looks like a project from some type of field study where time of collection could not be treated as fixed and simultaneous. We feel comfortable saying things like "Morning, Day 5" because we accept the noise around our fixed time points. Time seems to be a random effect nested within subject here. It also appears that time points are random within a visit block. I am sure there is some way to model this, but I doubt it of value here. Also with one value at each subject and time point, I doubt a simpler mixed model would converge.

The problem of interest is: when can we reject change from baseline is less than or equal to zero? First, subtract out the baseline for each subject. This leaves 6 time points for each subject, with six independent curves for change from baseline and a theoretical average curve. It seems like you have a function in mind with a calculated 95th prediction interval. I recommend a polynomial regression because it is computationally compact and you are not that interested in the fit of the model beyond the lower time values. Note I said prediction interval because this would apply to the observed change from baseline, not the mean change from baseline.

I would do a bootstrap with replacement by subject to retain the within subject association. There are 46656 possible samples of six subjects with replacement so 10000 bootstrap samples should give a range of values with not too frequent repetition. So for 10000 bootstrap samples, there would be 10000 curves each with its own lower 95% prediction interval, and from 6 to 36 values of time used to model the data. Back-calculate to find time from change from baseline =0. Use the median of the 10000 time values obtained from this process as your estimate.

  • $\begingroup$ Thanks for the response. Very interesting idea. Just to make sure I understand: when you say bootstrap with replacement by subject, you're suggesting choosing all measurements for a subject at once for the bootstrapped sample, correct? Also, in the last paragraph you mentioned to back-calculate time - did you mean to calculate X from the curve fitted to the bootstrap sample? Repeating this 10,000 times would give a distribution of X values, from which we could choose the median as our estimate. (I point this out because X is not actually time.) $\endgroup$
    – KirkD_CO
    Mar 4 '18 at 20:08
  • $\begingroup$ Yes, choose all measurements from a subject for n=6 sampled subjects. Yes, get each value of x from the curve fitted to the bootstrap sampl.e $\endgroup$
    – Georgette
    Mar 5 '18 at 21:31
  • $\begingroup$ Revisiting this now, admittedly some time later, and feeling a bit confused. For the problem at hand, it is still exactly as described. The suggestion was to do bootstrap sampling and compute a fit (e.g., polynomial regression) for each sample. Polynomial has a computable PI which can be obtained for each sample fit. For the estimate, take the median of the 10,000 computed 95% PIs. Correct? Questions: what if I fit with NLME? Or, what if I cannot compute a 95% PI for whatever method I use? Also, can I get a 95% CI in addition to the PI. $\endgroup$
    – KirkD_CO
    Jun 19 '18 at 1:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.