Others ways of feature selection aside from sparcity inducing linear algorithms and or random forests What are other ways of feature selection within a dataset $X$ for classifying a binary response variable $y$. I know such tools as logistic regression / linear SVMs with an l1 penalty, and sparse PCA. Then in the nonlinear category I only know of random forest / extra trees ensembles using feature importance metrics. 
What are other techniques, linear or otherwise that allow one to select a subset of features within $X$ for improving classification metrics. 
Aside from standard "off the shelf" techniques like the suggestions above what are more "sophisticated" or involved procedures for feature selection ? 
 A: In terms of unsupervised methods, there are some advanced sparse PCA methods that are more suitable for image classification problems. Examples are structured sparse PCA (Jenatton etal. 2010). Something that I have some experience with myself is sparse PCA with preserved sparsity (i.e., global/joint sparse PCA); a method that makes sure PCA selects the same features when the reduced dimension is larger than 1. You can find some code samples here. 
If you're interested in supervised methods, many use canonical correlation analysis (CCA) in a least squares framework to find a weight matrix for the features. This is essentially a feature selection scheme. 
Quick explanation of CCA: let's say you have N sample features of dimension d stored in a Matrix X and their corresponding N binary labels are stored in the matrix Y. CCA can provide a pair of weight matrices that minimizes norm(XW - YV), where W and V are the weight matrices. W can be viewed as a feature selection matrix. 
NOTE:
Y doesn't have to be binary when using CCA. Therefore, many have considered using CCA for multi-class classification. 
A: I have used an auto-encoder for dimensionality reduction. 
After the encoder is trained to stability, you can use the bottlenecked (last) hidden layer for obtaining non linear reduction in dimensionality.
