I have been searching for several days for a method that fits this description, though cannot find one. I'm pretty sure it must exist.
The problem (short version):
I'd like to run something like a CART, though instead of making splits to improve information/purity, it would make splits to maximise the average of the absolute value of the correlation between two variables in the daughter nodes. I'm pretty sure this is something that people do in cluster analysis, though I'm not acquainted very well with those methods.
Importantly, I'd need this method to be able to output a leaf/cluster membership at the observation level.
Any pointers would be appreciated.
The problem (long version):
We've run a randomised control trial with outcomes $Y$ and treatment $T$ (for the moment, you can consider these as being continuous). We also have surveyed each of the people on $K$ characteristics which are in matrix $X$.
Ideally, I'd like to know everyone's individual treatment effect, which of course is impossible. If we had that, we could prioritise treatment to those who are likely to benefit most. Heterogeneous treatment effect models aren't great, and the (best of class) methods I've been trying do not cross-validate well.
It may be a good second best solution to prioritise clusters of people who have the largest (expected) treatment effects. One way of clustering observations that are likely to have similar treatment effects would be to cluster them based on the correlation between $T$ and $Y$. Another method would be to cluster based on measured treatment effect in the daughter nodes, though then you'd need to specify some cost function to differentiate between cases of higher treatment effects with wide confidence intervals and lower TEs with tighter precision.
This may be misguided, and I'm very open to suggestions.