1
$\begingroup$

I have some data - about 100 values, maximum value 100.

However, values below 10 are just written as <10, exact values for these are not available.

How can I impute these data points. What value should I assume these data points to have? Should I take 5 (midway between 0 and 10)? If we assume normal distribution for whole data, average of these points would be around 7 (considering tail of normal distribution curve).

Thanks for your insight.

Edit: re discussion in comments:

enter image description here

For a perfect triangle with base between 0 and x, mean seems to be 0.633 times x (on checking data on spreadsheet), though I do not have mathematical proof.

$\endgroup$
8
  • 1
    $\begingroup$ What are you trying to use them to do? $\endgroup$
    – Glen_b
    Dec 10, 2019 at 4:53
  • $\begingroup$ What percentage of values are censored, "< 10" ? $\endgroup$ Dec 10, 2019 at 11:26
  • $\begingroup$ @SalMangiafico : About 5-7% of values are below 10. $\endgroup$
    – rnso
    Dec 10, 2019 at 12:23
  • $\begingroup$ @Glen_b-ReinstateMonica : I have to use this data as a continuous variable for determining group differences, correlations etc. $\endgroup$
    – rnso
    Dec 10, 2019 at 12:24
  • 2
    $\begingroup$ I'd be inclined to treat the data as what it is -- left-censored data. I wouldn't try to invent data, I'd use the data as it is. For some purposes you may want to use a parametric distributional model but you still have censored data in that context. $\endgroup$
    – Glen_b
    Dec 10, 2019 at 13:12

1 Answer 1

3
$\begingroup$

For some simple approaches with censored data, see USEPA, 2000, Guidance for Data Quality Assessment: Practical Methods for Data Analysis,EPA/600/R-96/084. Section 4.7. Note that this document was last revised 20 years ago.

They recommend that if less than 15% of the data is censored †, that a value of half the censored limit can be used ‡. For larger percentages, they recommend trimmed means or categorizing data and using a test of proportions. Among other approaches.

Also, I think it's worthwhile to think about what kind of data you have and what went into the censoring process. This may inform the approach you take.

__

† Here, they're considering environmental data, which I presume assumes the data are non-negative, left-censored, and the censoring is done at the detection limit of the chemical/physical analysis.

‡ It worries me some that your limit of censoring (10) isn't that many orders of magnitude away from the maximum observed value (100). Often, say with water quality data, you might have, say, a detection limit of 0.1 and a maximum of 10. This at least gives you a couple orders of magnitude between the detection limit and the maximum of the data. Because of this, it will often make no practical difference if the actual observed value was 0, 0.05, or 0.1. For your data, I wonder if the difference between 0, 5, and 10 would make a difference in interpreting data.

$\endgroup$
6
  • $\begingroup$ Can there be any formula/algorithm for value to be used given maximum recorded, minimum recorded and size of data/proportion censored? $\endgroup$
    – rnso
    Dec 10, 2019 at 17:14
  • $\begingroup$ I don't know of anything like that. $\endgroup$ Dec 10, 2019 at 18:24
  • $\begingroup$ Considering triangular shape of lower limit (for otherwise normally distributed data) 2/3 of lower limit rather than 1/2 may be more reasonable. $\endgroup$
    – rnso
    Dec 11, 2019 at 1:18
  • $\begingroup$ I don't understand what you mean. If what are saying is relevant to your question, you need to provide those details in your question. $\endgroup$ Dec 11, 2019 at 1:22
  • $\begingroup$ Pl see image in my question above which I have added as edit. $\endgroup$
    – rnso
    Dec 14, 2019 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.