Does a term exist for this concept in statistics? Suppose you have a dataset where the covariates are biomedical information for a given patient (e.g. height, weight, blood type, etc.) and the response variable is whether the patient has a certain disease. You are interested in making a classification model that will help screen patients in advance for this disease (i.e. save time and money).
Suppose you are able to create a decision tree that produces decent accuracy - but the "rules" produced by the decision tree are not very "helpful".
For example: if height between (150, 180 cm) and weight between (150, 200 lbs) then disease = true.
Suppose this rule proves to be a very accurate rule (i.e. when this condition is true, patients almost always have the disease) - but it's deemed too vague by the doctors to actually use. Instead, the doctors prefer to spend money/resources and physically screen patients.
Is there a term for this in statistics? An accurate model which is deemed too arbitrary to use?
 A: BLACK-BOX MODEL
That is the closest that I can think to what you describe. However, a "black-box model" is not necessarily useless. While many applications want to know how the model works, there are some situations where just getting the right answer is the crucial aspect.
The name comes from the idea of throwing data into a black box, so that seeing in is impossible, and it pops out an answer; the inability to see inside the black box to see how it makes its predictions is critical. One popular example of a black-box model is a neural network, which can give extremely accurate predictions (look at MNIST performance) but is hard to interpret.
A: You’re talking about making predictions not inference, but in case of inference we talk about practical significance. Statistical significance may help you to detect an effect that “significantly” differs from zero, but the effect may be so small that is has no practical significance, there’s no practical use of it.
A: Could this be an issue of precision and recall?
You are writing "when this condition is true, patients almost always have the disease" and call this accuracy. But usually one/I would not call that accuracy. The English Wikipedia has a large table in the "Precision and recall" article https://en.wikipedia.org/wiki/Precision_and_recall Here accuracy is, - as usual: (TP + TN) / (P + N), so your accuracy could approximately be rephrased as "when this condition is true, patients almost always have the disease AND when the condition is not true 'patients' almost never have the disease."
One should think that e.g., the precision or the recall is too low for the doctors. It may also be that the prevalence of the disease is very low, so that you can reach apparently high accuracy with just declaring everyone healthy. In screeening, perhaps the "False omission rate" is all what the doctors care about.
A: Is this a case of overfitting? Decision trees are prone to do that. Have you got enough data to use a held out test set to validate the model? If so, the doctors might be convinced by the results on the test set.
Alternatively, it might be a case where an executive commissions a model, but when it is delivered the in-house experts whose judgement it replaces refuse to use it. It would be good to have a name for this common phenomenon!
A: You state

Suppose this rule proves to be a very accurate rule (i.e. when this condition is true, patients almost always have the disease)

But that is often not the situation. If the broad boundary is very accurate, and if the detection means almost certainly that the patient has the disease, then this is a very good decision rule. The doctors will be happy with it.
The situation is more like below:

*

*Broad boundary:
This detects most cases, it is sensitive.
But, it is not accurate. Often the detection is false and the person is not sick.


*Narrow boundary:
This is accurate. If it makes a detection then often the person is also really sick.
But it is not sensitive. It is missing a lot of cases.


Is there a term for this in statistics?

There are many terms for this. See this wiki article on 'Sensitivity and specificity'.
