3
$\begingroup$

I need a word. We have ablation testing (or an ablation study, or ablation analysis) to quantify what happens if you remove a feature / component from a ML system. So one removes that feature/component, and see how much the performance decreases; and that says how important that feature/component was.

What do we call it when we replace some part of the system that was using an estimated feature with one that has perfect information, and see how much the performance improves? This allows us to estimate how much the quality of the estimated feature matters. using this information one can decide if it is worth spending effort on improving the subsystem that does that estimate (or if estimated feature is externally aquired, like weather forecasts, if it is worth looking for a better source).

$\endgroup$

1 Answer 1

1
$\begingroup$

Ceiling Analysis

You are precisely describing Ceiling Analysis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.