Given a non-elliptical cluster of points in a n-dimensional space I would like to get a distance function from the centroid of this cluster such that its "equipotential" surfaces has the same shape as the surface that encloses more or less tightly the points cloud (not overfitting the noise). So different distances from the center would result in scaled surfaces versions of the same shape. I guess I can achieve that for ellipsoid clusters by using the mahalanobis distance, but what about non-ellipsoid distances? Maybe a I should fit a mixture of gaussians and use the log-probability of generating a sample as being the distance to that sample? Is there a better way to accomplish what I want?
The centroid only works well for symmetric distributions.
The Mahalanobis distance will work well enough.
But you may be looking for SVMs instead.