One of the assumptions in the generalizability of machine learning algorithms is that observations should be IID.

But in many cases, observations come in natural clusters, in which observations are more similar than with observations in other clusters; furthermore, clusters bring information that is not modeled at the single observation level.

Examples are kids clustered in class/schools, patients in wards/hospitals. For linear modeling, Mixed Effects or Bayesian Hierarchical analysis are commonly used to account for the bias brought about by the loss of independence and the information implicit in clusters.

But how can adjust for this in other machine learning algorithms? Should one the clusters as variables and use a feature selection procedure to keep into account just the relevant clusters?

Are there other more accepted solutions?


closed as too broad by kjetil b halvorsen, whuber Apr 25 at 2:14

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.