TLDR Bootstrapping resulted in a crazy scatterplot. Totally clueless here.

I have a dataset (not going to say much about it, it is a bit confidential), running a Markov chain-based algorithm on the data. 458 data points in total.

Wanted to see the volatility of the metric I'm interested in (it is a ratio), decided to bootstrap my 458 data points and compare the results to the original rates, thereby seeing whether there is a relationship between rates and over/underestimations. (I hoped for something linear, so I could create a correctional multiplier). So I didn't simply do bootstrapping until convergence, just a large enough number of samples to see whether there is a relationship which I obviously miss if I stick to my original data and the ratio given one algorithm run.

enter image description here

Problem is, rates calculated from the bootstrapped samples exhibit a very peculiar behavior, not seen in other applications on similar datasets. It seems there are 3 distinct strands of points, one intermixed with a loose cloud of others. I wanted to explore the relationship b/w over/under and rate, but I'm clueless here.

enter image description here

(Besides delving into the data more,) do you happen to have any ideas what to go for?

Thank you.

  • $\begingroup$ What exactly is the problem? What does the plots show? Single sample from bootstrap should look like your data, doesn't it look like the initial data? $\endgroup$ – Tim Jan 30 '19 at 10:48
  • $\begingroup$ Looks to me like data with five curvilinear data clusters. I have seen this kind of thing associated with data truncation, although there are other causes that in effect result in clustering, still, most common for truncation. $\endgroup$ – Carl Jan 30 '19 at 11:16
  • $\begingroup$ @Tim, initial data was only one data point, since this is a metric from a Markov Chain. Other datasets exhibit more or less normal distributions after using same method. $\endgroup$ – KGYM Jan 30 '19 at 11:43
  • $\begingroup$ Then I don't understand what you are doing. You are taking samples of size 1 from dataset consisting 1 datapoint (this is bootstrap)? Then on your plot you should see only a single point, repeated the same number of times as number of your bootstrap replications. $\endgroup$ – Tim Jan 30 '19 at 11:51
  • $\begingroup$ @Tim, evidently this is not what I'm doing, since your version would result in all of the data points being in the same spot - and this is not the case. Carl understood my problem. I will update the question in a few minutes. $\endgroup$ – KGYM Jan 30 '19 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.