1
$\begingroup$

I'm working on a regression problem involving nutrient concentrations. The lab I'm getting my data from can measure up to 9000ppm of a particular nutrient. Beyond that, everything is reported as 9000pm. Would including these data points skew the regression, therefore I should remove them?

The model is CatBoost, which is decision-tree based.

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Are the nutrient concentrations predictor variables in. your model, or outcomes? $\endgroup$
    – EdM
    Commented Jul 19, 2022 at 17:10

1 Answer 1

Reset to default
1
$\begingroup$

No, removing those values would skew the regression, since you would be removing all the high values, so your results will be biased downwards. The correct way is to use models that allow for censoring.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you! I'll look into whether our methods are suited to censored data. $\endgroup$
    – Viv Crowe
    Commented Jul 20, 2022 at 22:33
  • $\begingroup$ @VivCrowe your outcomes are right-censored (you only have a lower limit to some values), typical of survival data. Buckley-James regression for right-censored data is one approach. Such regression models are implemented by the bj() function in the R rms package. The bujar package evidently extends that approach to regression trees and other methods, but I haven't used that package. $\endgroup$
    – EdM
    Commented Jul 21, 2022 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.