Which comes first - domain expertise or an experimental approach?

Question

In my organisation, we are embarking on an AI initiative where we try to identify business use cases and solve them using traditional ML algorithms.

However, our business users say that before they even take part in brainstorming, selecting, and reducing the feature space, they are asking the data folks to do a detailed scan and experiments and find out what are the most important and looks like important features through experiments...

Example: Let's say my data has 200 features and 30K rows. Our business team says that they will not be able to guide what the most relevant features to look at are, because they think this might bias the results. So, they want the data folks to find the important features through experiments.

Later, take these features and go to business team to check its relevance. Basically, no domain expert input until they get some confidence in what the algorithm outputs (for relevant features which has influence on the target variable).

Is this how it normally works in real-world AI projects? Is this a better approach to start with an AI project? Is there anything that we should be aware of?

Answer 1

This will probably be closed quickly as opinion-based, but here is a point you may want to consider.

200 features is a lot, and 30k rows is less than it sounds like. A "fishing expedition" to find relevant features is quite likely to overfit and select spurious features. The danger is that when you go to your domain experts with these features you "found" to be relevant, they may not push back. Instead, it's a very common human reaction to start telling stories about how these features are indeed useful, because we humans are very good at explaining stuff, even stuff that is simply noise.

Talking to your domain experts first will not completely avoid this problem, but it may reduce the number of wild goose chases.

You may be interested in my answer to "How to know that your machine learning problem is hopeless?".

Answer 2

John Elder in 2005 gave a (now classic) presentation called: "Top 10 Data Mining Mistakes". Number 4 in that list is: Listen (only) to the data.

Specifically for business environments where it is almost certain that we act using incomplete information (e.g. client priorities, financial and physical resources, legal framework, etc.) which affects our outcome of interest, ignoring prior knowledge can be very detrimental. At best we will duplicate work and/or produce trivial results; at worst we will get nonsensical "data-driven" findings. To be fully bearish on this as Twyman's law states: "Any figure that looks interesting or different is usually wrong".

Some points specifically to your question:

1. Finding "the important features through experiments" will entail a full experimental design. If they are will to invest the time and money to it, go for it. Also familiarise yourself with the "Analysis of Observational Data" (yes, it is "unsexy" to read about survey analysis but treating a biased sample as being random is catastrophic.)
2. This is good point in an analysis project life to consider applications of Bayesian inference as a way to formalise modelling assumptions. Simply put, if we have 200 features some have to be more likely to affect our outcome than others; use priors to encapsulate that.
3. Respect the physics of the environment you operate in (until you don't have too). Signal strength diminishes as distance increases; average client in Luxenbourg has more disposal income than one in Bulgaria. Use well-established hypotheses as priors/starting points of the analysis; update wisely.
4. Read the literature. Really, this is not just an academic activity. Very often we can get reasonable answers by curated resources if we make formal searches. Similarly, if there is nothing available this is even more of a reason to revisit points 1 & 2 in greater detail.

To recap: absolutely talk to domain experts (or work hard to become a budding one yourself). It can save you a world of pain; if anything downstream you will have a better idea on how to present your findings and what were/are usual contention points.

Answer 3

The problem you are dealing with is a selection of variables problem, and so standard principles and methods apply. In particular, if you have a large number of initial variables/features to select from, there is a danger of overfitting if you fail to adopt appropriate methods that account for multiple comparisons. In the case where you have an exogenous source of information about the variables from subject matter experts, this can be used for prior information to guide your selection method --- e.g., to narrow down the model/comparisons you will make.

You are correct to have serious misgivings about the proposal made to you. The idea of first using statistical methods to find relevant features, and then giving this information to the business team to inform their feedback sounds like a terrible idea to me. By doing this you take a potential form of a priori exogenous information and you bias it heavily by feeding in your statistical results first. It will certainly bias the feedback of the business team and so whatever information they give you after that is likely to be contaminated and useless. If you then narrow your model/fitting to focus on the features/variables they "confirm", this will effectively over-weight the data in your original model fit (since they will largely just feed back the posterior results as prior information) and you will almost certainly overfit your model. I recommend reading Gelman and Loken (2013) on the "garden of forking paths" that occurs when making research/modelling choices.

Answer 4

There are two aspects here causal inference and explainability.

From a causal inference perspective, domain expertise should guide the process of building relevant factors on a given purpose, targets, that are really linked and not just correlations explored or discovered by data scientists. Inference and Intervention: Causal Models for Business, by Ryall-Bramson (2013), routledge provides set of case studies on how domain knowledge can guide building causal models.

In purely machine learning projects, bottom line is model's performance. So called feature discovery and extraction is usually performed without domain experts input, and best performing model is generated by data scientist. Domain experts involvement in this approach appears to be in ensuring to set to right business target in machine learning models. If it is a regulated environment or a requirement is there, then the machine learning model explainability work might require intervention from domain expert, see Interpretable Machine Learning.