I recently stumbled upon Y-aware PCA in the blog of win-vector.
They describe how PCA can be adjusted not to explain variation in $X$ but covariation of $X$ and $Y$.
This is explained for the case where $Y$ is continuous. How could one do this in the case where $Y$ is binary? Partial-least-squares (PLS) does something very similar. But to my knowledge it can only be used for regression. What I would like to do is to preprocess data in this Y-PCA style and then apply some other classifier (tree-based e.g.).
Is there any reference for this? Any (best open-source/R implementation) on the web?
EDIT: What I did so far is to apply partial least-squares regression with a binary output and I just keep the projection matrix, say $T$. Now I try other classifiers using the $XT$ instead of $X$ as explanatory variables.