How would Y-aware PCA for binaries look? I recently stumbled upon Y-aware PCA in the blog of win-vector.
They describe how PCA can be adjusted not to explain variation in $X$ but covariation of $X$ and $Y$.
This is explained for the case where $Y$ is continuous. How could one do this in the case where $Y$ is binary? 
Partial-least-squares (PLS) does something very similar. But to my knowledge it can only be used for regression. What I would like to do is to preprocess data in this Y-PCA style and then apply some other classifier (tree-based e.g.).
Is there any reference for this? Any (best open-source/R implementation) on the web?
EDIT: What I did so far is to apply partial least-squares regression with a binary output and I just keep the projection matrix, say $T$. Now I try other classifiers using the $XT$ instead of $X$ as explanatory variables.
 A: A better link to the blog on "Y-aware PCA" is here. The authors of that blog have an R package vtreat that implements this and other approaches to conditioning variables before analysis.
As noted in some comments, Y-aware PCA is related to partial least squares (PLS). It weights predictor variables according to their single-variable relations to the outcome variable, which is similar to the first step in PLS. Y-aware PCA then uses those weighted individual predictor variables, rather than their original standardized values, as the input to PCA. PLS, in contrast, uses the weighted sum of those predictor variables as its first component and constructs further orthogonal components by successive regression against residuals (e.g., ISLR, pp. 237-238).
One of the vignettes shows those authors' approach to the binary-outcome issue in their package:

categorical/logical y’s are treated as 0/1 indicators

That might not seem terribly satisfying. As this review on PLS puts it:

... applying a regression method designed for continuous responses to categorical responses or performing dimension reduction with survival data without taking censoring into account is unappealing, although it is reported to give good results in many cases.

The review goes on to cite work in which Cox or logistic regression coefficients were used instead of linear regression coefficients for PLS. That might be a reasonable extension of "Y-aware PCA" to the binary outcome situation.
