Why is the variable importance metric suggested by Breiman specific only to random forests? In the Random Forest paper they describe a nice way of measuring a variable importance - take your validation data, measure error rate, permute the variable and re-measure error rate.
Question - why is that method specific to Random Forests? I understand that in other classifiers (SVM, LR, etc.) we don't have the concept of OOB, but we certainly can use a regular train-validation split.
What am I missing here? Why isn't this method a common practice?
 A: Any bagged learner can produce an analogue of Random Forests importance metric.
You can't get this kind of feature importance in a common cross-validation scheme, where all the features are used all the time.
A: Random Forrest and other techniques that incorporate bagging are using the fact that the bootstrap sample that is drawn for the current tree excludes some data points, the so-called Out-Of-Bag samples (OOB). Since these samples are not used to build the current tree, they can be used to evaluate it without the risk of overfitting. With other supervised learning techniques that usually do not suffer from instability as much as decision trees (e.g. SVM), you usually
do not draw bootstrap samples and thus you can not estimate variable importance in this way. 
However, the approach of training a model with different subsets of variables and evaluate their performance using k-fold cross-validation is also perfectly valid and called Wrapper approach in the literature. For instance, a popular feature selection technique with SVM is recursive feature elimination (see https://pdfs.semanticscholar.org/fb6b/4b57f431a0cfbb83bb2af8beab4ee694e94c.pdf) 
