2
$\begingroup$

I have a set of 1000 datapoints of measured concentrations that may include up to 300 values which are censored (below the detection limit that the lab could reliably measure). The range of detection limit values vary, such as <2, <3, <7 etc. My data is neither normal or log-normal, and I've used non-parametric tests to analyze it so far.

The information in the environmental-research literature about the use of kaplan-meier, or ROS estimators for real left-censored data primarily only compares overall statistics (i.e. mean, median, std. dev) between these different estimator methods.

I would like to use KM to generate individual values for those of my results which are below the detection limit. To date I've relied on whatever stats software may be available to me, but I cannot find this option presently.

Edit: What are the steps to generate values for the left-censored data (using KM)?

Is there a programs/software that I could apply to my dataset and then impute values based on KM? Ultimately I am interested in using these generated values for further multivariate analysis of my dataset, and thus need new individual values (as opposed to overall mean, median etc).

Any comments would be helpful. Thank you.

$\endgroup$
  • $\begingroup$ Assuming nonnegative, can't you just invert the observations and apply the KM estimator to that? It is my guess is that this is equivalent to deriving and using the KM estimator for left censored data, but I'm not sure. $\endgroup$ – guy Nov 25 '13 at 1:40
  • $\begingroup$ @Oleic I am a student, working on my master thesis and I have the same problem as you mentioned some time ago. Could I ask how did you solve the situation at the end? Thank you for your answer. $\endgroup$ – user62339 Dec 8 '14 at 10:25
  • $\begingroup$ @AndrejaK I tried a number of different things, including the suggestions made here, and ended up building off the comments of user31668 below. $\endgroup$ – Oleic Feb 25 '15 at 9:10
1
$\begingroup$

Yes it is possible to use the Kaplan method to estimate left-censored data. The Wiki article is actually pretty decent Check it out

$\endgroup$
  • $\begingroup$ I had a chance to review it. I follow the way survival is estimated. However could you or someone explain how I would use this approach to generate new values for my left-censored data points? $\endgroup$ – Oleic Nov 24 '13 at 21:04
  • 1
    $\begingroup$ @Oleic the Kaplan meier estimator does not extrapolate beyond your data. What you are looking to do is impute points. You will need to specify some model for the tails of your distribution (a common one is an exponential). Non parametrics cannot do this for you as anything is possible outside of your actual observations unless you use your contextual knowledge to bound things....but then you're no longer strictly nonparametric. $\endgroup$ – user31668 Feb 11 '14 at 19:33
  • $\begingroup$ @Eupraxis1981 - thank you for confirming, this makes sense. I tried assuming log distribution to impute points using ROS as I see it often used with `environmental' data. I used an existing software for the imputations which does not include exponential. Can you describe how one can do the imputation on their own? That is do I just plot an exponential regression on the existing data and then find points off the line? $\endgroup$ – Oleic Feb 14 '14 at 4:21
  • $\begingroup$ @Oleic the exponential is not a good model for a left tail for strictly positive data. ROS or imputation using lognormal or weibull is better, so use one of those if your software has it. This book may be useful if you haven't already run across it in your literature search: pubs.usgs.gov/twri/twri4a3/pdf/twri4a3-new.pdf $\endgroup$ – user31668 Feb 14 '14 at 4:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.