# Finding most similar training samples for a given ML model output

Let's say you have trained a regression model. Now, in production, you get a new input, and you want to identify K samples in the training set that are the most similar to the new input as far as the model is concerned.

I do realize that the solution to this depends entirely on the similarity metric, and that the appropriate similarity metric is ill-defined - i.e., how do we know that our similarity metric is doing a good job? But we probably could tell it is doing a good job qualitatively, i.e. know-it-when-we-see-it.

But let us start from a simple example. Let's say our model is a linear regression, y = w0*1 + w1*x1 + w2*x2 + w3*x3. It seems that considering the distance between [w0*1, w1*x1, w2*x2, w3*x3] for the two samples would be a good idea:

1. It normalizes the features according to feature importances
2. "Similar" inputs would produce similar outputs

It certainly seems better than measuring the distance between [x1, x2, x3] or [normalized(x1), normalized(x2), normalized(x3)], as these do not take into account any information learned by the model, apart from maybe feature importance. It also seems better than simply comparing [y1] with [y2] - samples in very different parts of the feature space can have similar outputs, but it does not seem right to say that the samples are "similar", i.e. the model gave them similar outputs for similar reasons.

Now let's say we have a more complex example, where our model is XGBoost. We could follow a similar approach - for every sample, break down the final model output into a sum of contributions from each feature (e.g. using the approach used by eli5.explain_prediction()), and then check the distance between vectors of such contributions for the new input and training set samples.

1. Does the approach seem sensible?
2. Am I reinventing the wheel here, i.e. are there any known existing approaches for interpreting ML models by finding most similar training set samples according to the model?
• What is "similar" to the ML model depends on the model -- how does the model "think about" the data? Are you asking for a generic method to interpret similarity, or are you specifically interested in how to assess similarity according to XGBoost models?
– Sycorax
Jul 24, 2018 at 12:55
• @Sycorax I am specifically interested in XGBoost, although it seems that a method that is applicable to XGBoost might also be applicable to other models. Jul 24, 2018 at 15:51

Found a recent paper that presents a similar approach to the one I described in the question: "Consistent Individualized Feature Attribution for Tree Ensembles", Lundberg et.al., KDD 2018.

They use Shapley values to explain the model prediction for each sample as a sum of contributions from each feature, and then compare the distances between the resulting vectors to find samples that are similar according to the model. The results seem to be fairly intuitive and interpretable: So the approach in the question appears to be both sensible and used elsewhere.

If you are looking to pick 'n' closest samples from the training data set that is similar to an out-of-sample observation, why not simply use the features you have to get a distance with each training observation - euclidean, manhattan or whatever (depends on the feature types)? The training sample with the smallest distance to the new observation would be the most similar.

• Consider the case of a linear regression, where y = 1000*x1 + 0.001*x2 + 0.001*x3 and four samples [10, 10, 10], [10, 10, 0], [11, 10, 10], [0, 20, 10^7]. If we do what you propose, ignoring what the model learned, we would probably say that sample 1 is most similar to sample 3. If we just look at model outputs, we would say that sample 1 is most similar to sample 4. Whereas intuitively it would be more sensible to say that according to the model, sample 1 is most similar to sample 2. Jul 20, 2018 at 9:28
• I'm not too sure about your definition of similarity. From my point, any two data points with the same characteristics/features are similar. You might still have completely different features for two data points, yet obtain similar predictions. But that depends on your model. In that case, why not include the response variable (or predicted value) into the distance matching? Jul 20, 2018 at 9:55
• Imagine a situation where we have 100 irrelevant features (according to the model), and only 1 highly relevant feature. Even if we include the output in the distance metric, and even if we normalize all features as well as the output, the distance metric will be dominated by irrelevant features. Jul 20, 2018 at 10:10