What I would suggest as a solution is Penalized Mahalanobis distance. You can see this blog post for details http://stefansavev.com/blog/better-euclidean-distance-with-the-svd-penalized-mahalanobis-distance/. You can also check "The Elements of Statistical Learning", by Hastie et. al. in particular the sections on ridge regression (it is related) and look up in the index Mahalanobis Distance

What I would suggest as a solution is Penalized Mahalanobis distance. You can see this blog post for details http://stefansavev.com/blog/better-euclidean-distance-with-the-svd-penalized-mahalanobis-distance/. You can also check "The Elements of Statistical Learning", by Hastie et al. in particular the sections on ridge regression (it is related) and look up in the index Mahalanobis Distance