your question and discussion with Scortchiare very thought-provoking.
I'd like to share my perspective on the difference between these two methods.

Permutation Importance and Drop Column Importance are associated with two types of data collection problems. Permutation importance provides an estimation for the performance loss when the data is collected wrongly or manipulated, like a column being shifted one row upward/downward; drop column importance gives us an insight into what will happen if we don't collect or use this feature.

The choice depends, among other factors, on the prospective data issues.

Your question and discussion with Scortchiare are very thought-provoking. I'd like to share my perspective on the difference between these two methods.

Permutation Importance and Drop Column Importance are associated with two types of data collection problems. Permutation importance provides an estimation for the performance loss when the data is collected wrongly or manipulated, like a column being shifted one row upward/downward. Drop column importance gives us an insight into what will happen if we don't collect or use this feature.

The choice depends, among other factors, on the prospective data issues.