3
$\begingroup$

I estimated the breakpoint for a piecewise linear model (one intercept, two slopes that meet at the breakpoint) by minimizing the deviance using an optimizer (optim in R). I noticed that the estimate was near 2.0, 4.0, and 6.0, depending on the initial conditions, indicating there were local optima. So I used simulated annealing to estimate the breakpoint (optim(..., method="SANN", ...)), and became reasonably confidence that I had found the globally best estimate at bp = 6.0. I just finished bootstrapping the breakpoint estimates (my data are nested within subjects, and I had to apply simulated annealing to each bootstrap estimate, so the whole thing took a couple days), and found that:

1) The bootstrap distribution is bimodal, with modes at 2.0 and 6.0.

2) The mean of the bootstrap distribution is 4.0, and the median is 3.5, whereas the parameter estimate was 6.0.

Can someone help me to understand my results? I want to say that the original estimate of 6.0 is an artifact of my sample, and that the bootstrap mean of 4.0 is a better estimate of the breakpoint, but I'm not sure if this is the right interpretation. Also, placing the breakpoint at 4.0 produces conditional means that agree almost exactly with a LOESS curve using default values (a breakpoint at 6.0 does not agree as well).

--

Edit: Adding a graph.

Red line = original fit with bp = 6.0. Black solid = fit with bp = 4.6. Black dotted = LOESS. Open circles = means. Models were fit to full data set (too many data points to show, and yes, I tried high density plotting methods as well), not means.

Figure notes: Red line = original fit with bp = 6.0. Black solid = fit with bp = 4.6. Black dotted = LOESS. Open circles = means. Models were fit to full data set (too many data points to show, and yes, I tried high density plotting methods as well), not means.

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ So what is there to not understand? You have a non-smooth problem, so the bootstrap distribution does not have to be a nice Gaussian (as it would be in the mean of i.i.d. data and other simple problems). Check dx.doi.org/10.1002/cjs.5550340103 for a good list of issues you may encounter with the bootstrap. $\endgroup$
    – StasK
    Commented Feb 17, 2012 at 20:10
  • $\begingroup$ Thanks for the reference @StasK - I am reading through it. I was not so concerned about the shape of the bootstrap distribution, but instead that it wasn't centered around the original estimate. What I wanted to know was whether I could consider the bootstrap mean of 4.0 to be a better estimate of the breakpoint than the original estimate of 6.0. $\endgroup$
    – lockedoff
    Commented Feb 17, 2012 at 20:54
  • $\begingroup$ It seems to me that you may have a single breakpoint at about 7.5, but it is a breakpoint in both slope and intercept. If so, the model is misspecified. Your bootstrap is finding two alternatives that are "close." Also, bootstraps with little data are of dubious, but not necessarily no, value... $\endgroup$
    – jbowman
    Commented Feb 18, 2012 at 16:29

1 Answer 1

Reset to default
5
$\begingroup$

To quote from the intro in the Efron/Tibshirani text on the subject:

The message of this book can therefore be summarized by paraphrasing Tukey: "The bootstrap, like a shotgun, can blow the head off any problem if the statistician can stand the resulting mess''.

It sounds like you're trying to make inference based on the assumption that the asymptotic bootstrap distribution of the estimated breakpoint is unimodal. What if the "true trend", in fact, had two break points?

Improve this answer
$\endgroup$
4
  • $\begingroup$ it's possible that there are two breakpoints, but I think it's more likely that subjects are heterogeneous in their single breakpoint location. I'm mostly basing this statement on the fact that a single-breakpoint model produced almost the exact same curve as a LOESS model. I'm more trying to understand whether the bootstrap mean is a better estimate than the original estimate, given the information I've provided. $\endgroup$
    – lockedoff
    Commented Feb 17, 2012 at 20:45
  • 1
    $\begingroup$ If there are two breakpoints, then a mean between the two would be a foolish thing to estimate. I would be very curious to see a scatterplot of the data. $\endgroup$
    – AdamO
    Commented Feb 17, 2012 at 21:26
  • $\begingroup$ I might have misunderstood your original answer. By asking whether "the 'true trend'... had two break points," were you suggesting bimodality? If so, I would agree with you. I thought you were asking something else. I've added a plot - it might not be too helpful. $\endgroup$
    – lockedoff
    Commented Feb 17, 2012 at 22:13
  • 1
    $\begingroup$ It is highly doubtful that the breakpoint even exists. If that's the case, the bootstrap will give answers reflecting that. $\endgroup$
    – Frank Harrell
    Commented Feb 18, 2012 at 0:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.