As I've been trying to get a grasp on the field of interpretable ML I have encountered many interesting papers and methods, but I lack the perspective to determine their importance to the field as a whole. To those with a better perspective: What are the most important/seminal/popular methods in the field of interpretable ML and what are the 'first principles' that make them so?
I think there are three big categories of interpretation methods:
Replace the complex black box model with an interpretable model that approximates the black box model reasonably well. The surrogate model can either explain the black box model as a whole, or parts of it, even at the level of individual predictions. LIME  is an implementation of the idea that you replace the black box prediction function with an interpretable model (for example a linear regression model) at the level of an individual data point and its neighborhood.
Some methods try to summarize the average effect a feature has on the prediction, for example, partial dependence plots (PDP) , accumulated local effect plots  and individual conditional expectation (ICE) curves. They usually summarize the feature effect averaged over the whole data, except for ICE curves which show the effects on an individual level. I would also see methods like counterfactual explanations here (e.g. "If feature x2 would be increased by 1 point, the predicted probability would increase by 5%") as describing the effect of features for an individual prediction.
Feature importance ranks the features by how much the model relies on them for making predictions. This can be formulated by how much the feature contributed to the performance of the model , but also by how much of the variance of the predictions is explained by a feature.
There are some further properties by which you can distinguish methods for interpretable machine learning:
- Scope: Global vs. Local: Is the interpretation for the entire model? Or maybe just for an individual prediction? For example, LIME produces local explanations while PDP produces global ones.
- Model-agnostic vs. Model-specific: Does the interpretation method only work for certain models because it relies on the model-specific structure?
- Type of data: It depends on the data type which interpretation method can be used. PDP doesn't make sense when the inputs are image pixels. I would see tabular, text and image data as three common classes.