Are there popular dimensionality reduction tools for classification type supervised learning? I am familiar with PCA as a linear transformation in order to align the axis of the IV space with the directions of maximal variance in order to possibly be able to reduce the dimensionality of the problem. Are there similar techniques (or can one use PCA itself) when facing a binary classification problem? Like a logistic regression? I.E. Suppose we have a dependent variable we want to predict which only takes values 0 and 1 and we have a lot of features from which to predict, but maybe there are "principal components" in the high dimensional feature space which carry the most explanatory power.
 A: Yes, you can use PCA retaining only certain components as pre-processing step for classification. It is actually pretty common in chemometric classification. 
However, I prefer PLS as it also uses your class labels and tries to find good separation (which PCA doesn't). 
In both cases, keep in mind that this is a data-driven pre-processing and validation of the final model needs data independent also of these pre-processing steps.
A: I like Information Gain, but there are others.
I found this paper (Fabrizio Sebastiani, Machine Learning in Automated Text Categorization, ACM Computing Surveys, Vol. 34, No.1, pp.1-47, 2002) to be a good theoretical treatment of text classification, including feature reduction by a variety of methods from the simple (Term Frequency) to the complex (Information-Theoretic).

These functions try to capture the intuition that the best terms for ci are the
  ones distributed most differently in the sets of positive and negative examples of
  ci. However, interpretations of this principle vary across different functions. For instance, in the experimental sciences χ2 is used to measure how the results of an observation differ (i.e., are independent) from the results expected according to an initial hypothesis (lower values indicate lower dependence). In DR we measure how independent tk and ci are. The terms tk with the lowest value for χ2(tk, ci) are thus the most independent from ci; since we are interested in the terms which are not, we select the terms for which χ2(tk, ci) is highest.

These techniques help you choose terms that are most useful in separating the training documents into the given classes; the terms with the highest predictive value for your problem.  
I've been successful using Information Gain for feature reduction and found this paper (Entropy based feature selection for text categorization Largeron, Christine and Moulin, Christophe and Géry, Mathias - SAC - Pages 924-928 2011) to be a very good practical guide.
Here the authors present a simple formulation of entropy-based feature selection that's useful for implementation in code:

Given a term tj and a category ck, ECCD(tj , ck) can be
  computed from a contingency table. Let A be the number
  of documents in the category containing tj ; B, the number
  of documents in the other categories containing tj ; C, the
  number of documents of ck which do not contain tj and D,
  the number of documents in the other categories which do
  not contain tj (with N = A + B + C + D):


Using this contingency table, Information Gain can be estimated by:

This approach is easy to implement and provides very good Information-Theoretic feature reduction.  
You needn't use a single technique either; you can combine them.  Ter-Frequency is simple, but can also be effective.  I've combined the Information Gain approach with Term Frequency to do feature selection successfully.  You should experiment with your data to see which technique or techniques work most effectively.  
