2
$\begingroup$

Suppose you have a dataset where the covariates are biomedical information for a given patient (e.g. height, weight, blood type, etc.) and the response variable is whether the patient has a certain disease. You are interested in making a classification model that will help screen patients in advance for this disease (i.e. save time and money).

Suppose you are able to create a decision tree that produces decent accuracy - but the "rules" produced by the decision tree are not very "helpful".

For example: if height between (150, 180 cm) and weight between (150, 200 lbs) then disease = true.

Suppose this rule proves to be a very accurate rule (i.e. when this condition is true, patients almost always have the disease) - but it's deemed too vague by the doctors to actually use. Instead, the doctors prefer to spend money/resources and physically screen patients.

Is there a term for this in statistics? An accurate model which is deemed too arbitrary to use?

Improve this question
$\endgroup$
2
  • $\begingroup$ I am not sure there's a statistical concept that maps on to this one to one. We all know that it is easy to make correct statements that are not informative (e.g. politicians can be evasive or you will die or even that most people survive this disease). Depending on context and expected style and who is talking or writing (student? professional? self-appointed expert, lay person?) I might describe that as correct but not informative, empty, vacuous, banal, platitudinous and so forth. $\endgroup$
    – Nick Cox
    Commented Oct 20, 2021 at 14:57
  • $\begingroup$ "Suppose this rule proves to be a very accurate rule (i.e. when this condition is true, patients almost always have the disease)" Did you mean instead 'when the patient has the disease then almost always the condition is true'? $\endgroup$
    – Sextus Empiricus
    Commented Oct 20, 2021 at 17:27

5 Answers 5

Reset to default
2
$\begingroup$

BLACK-BOX MODEL

That is the closest that I can think to what you describe. However, a "black-box model" is not necessarily useless. While many applications want to know how the model works, there are some situations where just getting the right answer is the crucial aspect.

The name comes from the idea of throwing data into a black box, so that seeing in is impossible, and it pops out an answer; the inability to see inside the black box to see how it makes its predictions is critical. One popular example of a black-box model is a neural network, which can give extremely accurate predictions (look at MNIST performance) but is hard to interpret.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Decision trees are often quoted as the contrary of a blackbox model. They are especially easy to interpret. I think in the OP, the problem arises from squeezing a lot of highly correlated and maybe unintuitive features into a model without caring about feature engineering. Maybe if the same features would be plugged into a linear regression, the resulting model would look similarly unnatural. Just speculating... $\endgroup$
    – Michael M
    Commented Oct 5, 2021 at 17:23
  • $\begingroup$ @MichaelM I agree that the stated model is not a good example of a black-box model, but the idea of "accurate predictions, hard/impossible interpretation" is exactly what a black-box model means. $\endgroup$
    – Dave
    Commented Oct 5, 2021 at 17:24
2
$\begingroup$

You’re talking about making predictions not inference, but in case of inference we talk about practical significance. Statistical significance may help you to detect an effect that “significantly” differs from zero, but the effect may be so small that is has no practical significance, there’s no practical use of it.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Statistical vs practical significance is important, certainly, but this seems rather unrelated. $\endgroup$
    – Dave
    Commented Oct 5, 2021 at 21:26
1
$\begingroup$

Could this be an issue of precision and recall?

You are writing "when this condition is true, patients almost always have the disease" and call this accuracy. But usually one/I would not call that accuracy. The English Wikipedia has a large table in the "Precision and recall" article https://en.wikipedia.org/wiki/Precision_and_recall Here accuracy is, - as usual: (TP + TN) / (P + N), so your accuracy could approximately be rephrased as "when this condition is true, patients almost always have the disease AND when the condition is not true 'patients' almost never have the disease."

One should think that e.g., the precision or the recall is too low for the doctors. It may also be that the prevalence of the disease is very low, so that you can reach apparently high accuracy with just declaring everyone healthy. In screeening, perhaps the "False omission rate" is all what the doctors care about.

Improve this answer
$\endgroup$
1
$\begingroup$

You state

Suppose this rule proves to be a very accurate rule (i.e. when this condition is true, patients almost always have the disease)

But that is often not the situation. If the broad boundary is very accurate, and if the detection means almost certainly that the patient has the disease, then this is a very good decision rule. The doctors will be happy with it.

The situation is more like below:

  • Broad boundary:

    This detects most cases, it is sensitive.

    But, it is not accurate. Often the detection is false and the person is not sick.

  • Narrow boundary:

    This is accurate. If it makes a detection then often the person is also really sick.

    But it is not sensitive. It is missing a lot of cases.

example

Is there a term for this in statistics?

There are many terms for this. See this wiki article on 'Sensitivity and specificity'.

Improve this answer
$\endgroup$
0
$\begingroup$

Is this a case of overfitting? Decision trees are prone to do that. Have you got enough data to use a held out test set to validate the model? If so, the doctors might be convinced by the results on the test set.

Alternatively, it might be a case where an executive commissions a model, but when it is delivered the in-house experts whose judgement it replaces refuse to use it. It would be good to have a name for this common phenomenon!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.