Suppose I have $n$ datapoints in $p$-dimensional space, and the $p$ variables are highly heterogenous. That is, there is no natural way to combine them. Some are ordinal, some one-hot, some continuous, etc. It is not clear which of them is more 'important,' so I have no natural way of weighting them.
I would like to cluster these points or embed them using t-SNE or UMAP or something similar. But, there is no natural definition of affinity/distance between points here. I could Z-score each variable and then use Euclidean distance, but I do not get an optimal clustering because all the variables are weighted equally, despite their 'importance' to me not being equal.
So far, the problem is poorly defined, it is not even clear what I mean by an 'optimal' or 'importance.' I think that I can make it a bit more well-defined by using another variable, which I will call $y$. Let $y$ to be an outcome I really care about, like mortality. I would like to use $y$ to learn a metric between points in my $p$-dimensional space.
As a general example, perhaps I could want my metric to be such that $y$ is somehow smooth with respect to a nearest neighbor graph induced by the affinities between points. Perhaps this would allow me to learn a notion of affinity.
More specifically, suppose I have medical records dataset and I am trying to learn different phenotypes of myocardial infarction. Then my $p$ variables would be from the electronic medical record, whereas 'had an MI' would be my $y$. I would want to use this approach to cluster individuals who have a 'similar kind' of heart attack.
Is there any work on learning a metric between points that somehow takes into consideration an 'outcome?' It is somehow unsupervised learning 'with respect' to some kind of outcome.