Can Machine Learning Models Recover "Experimental, Design and Hierarchical Structures" Within the Data?

Question

Can Machine Learning Models Recover "Experimental, Design and Hierarchical Structures" Within the Data?

At times, real world data can contain "embedded structures" - these structures are sometimes "controlled" (e.g. variation in chemistry exam scores from different countries) and sometimes they are "uncontrolled" (e.g. variation in chemistry exam scores within the same class). In the context of statistical modelling, these "structures" can often assist and guide us - by encouraging to analyze data and build separate models for data aggregated at different structural and sub-structural levels. We may even be interested in estimating the average "effects" that these structures have on our data - treating these effects themselves as random variables. For more information , please consult the following topics:

• Random, Fixed and Mixed Effects
• Hierarchical Models
• Nested Models
• Multilevel Models

However, I have the following question: Without being "told" about the existence of these structures in the data, can Machine Learning Models (e.g. Random Forest, Gradient Boosting) still perform well by latently modelling the effects of these structures?

If they can do so - this could be beneficial in these sense that Machine Learning Models might be able to "detect" convoluted structures embedded in the data (e.g. 3rd Level: by ZIP Code, School and Faculty) that might exist and have substantial effects - but are currently unknown to the researchers. However, I can imagine that even if the Machine Learning Models are able to make reliable predictions in the presence of detailed hierarchical structures in the data, they will not be able to identify these structures as easily - ultimately preventing a deeper understanding of such structures. I suppose that standard clustering methods might be applicable in recovering these structures and then training Machine Learning Models on each of these clusters - but the clusters run the risk of being sparse and uninformative. This leads me into believing that it still might be ideal to prefer classical statistical models where these kinds of effects and structures are directly modelled, instead of being passively analyzed in the background.

Can someone please provide some comments on this?

Thanks!

Answer 1

Oh yes, that can happen. And it can have majorly adverse and unlooked-for impacts.

We may not feed a job candidate's ethnicity into the ML system that parses all submitted CVs for equity reasons - but the ML tool may still use the applicant's ZIP code, school or other information to give a low rating, and these pieces of information may be highly correlated with ethnicity or other socioeconomic factors that we did not want to use.

Here is the first hit I got googling for "Amazon hiring ML discrimination", which was a hot story back in 2018. Note that applicants' gender was not supplied to the AI, but it still picked up on CV entries like "women’s chess club captain", or, more problematically for this question, degrees from all-women colleges. The result was that women were given lower rankings, because the system had been trained with historical data, which had a preponderance of successful men.

Note that for present purposes, it really doesn't matter whether our output reflects prior discrimination in training data, or true underlying differences correlated with these factors. The key aspect is that ML methods can indeed easily pick up groupings or other latent information that we do not want it to use.

Thus, yes, ML can recover this kind of latent information (and we may not want it to). It typically needs more data to do so than a "classical" model where we can supply this kind of context directly. Not every problem needs an ML solution, and feeding information we have into a tool will usually improve matters. (But beware of feeding in too much information, lest you fall into overfitting.)