A student of mine developed a heuristic supervised machine learning algorithm for highly multivariate data. It seems to work pretty well, and once the model has been derived from the training data set, we can represent it as
Where $X$ is the $n \times k$ sample matrix with $n$ samples and $k$ variables, and $L$ is a $k \times m$ loading matrix. The resulting $X'$ matrix is related (or can be understood) just like principal components matrix in a PCA, or the variates matrix in a PLS-DA.
The $X'$ matrix is then used for classification, and the assigned class is simply based on euclidean or Mahalanobis distances of the $m$-dimensional sample representations from the class centroids.
What would be, in your opinion, the best way of determining variable importance, such that we could sort the $k$ variables according to their contribution to the decision making process?