There are a number of helpful posts on subset selection (or feature selection) with continuous or binary response variables (i.e., here). However, I've not been able to find any posts on subset selection with categorical response variables.
Would discriminant function analysis be a promising starting point for subset selection with a categorical response variable? And are there any standard approaches?
You can train a classifier (e.g. multi-class SVM) with sparsity inducing regularization including $\ell_1$ or $\ell_1/\ell_2$ to ignore dimensions (or groups of dimensions) in the original feature vector that carry little information, as far as the classification task is concerned.
Let $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^n$ denote a set of $n$ labeled training examples where $y_i \in \{1, \dots, K\}$. The training objective of an $\ell_1/\ell_2$ regularized max-margin classifier with hinge loss (multi-class SVM) can be defined as follows: $$ O(w) = \lambda \sum_{g \in G} \sqrt{||w_g||^2} + \sum_{i=1}^n \left( \max_{y=1}^K \left( w \cdot \Phi(x_y, y) + \Delta(y, y_i) \right) - w \cdot \Phi(x_i, y_i) \right) \tag{1} $$ where $w = (w_1; w_2; \dots; w_K)$ is the classifier and $\Phi(x, y)$ is a $Kd$-dimensional feature vector with $K$ blocks of length $d$ each, where all the blocks are zero except for the $y$-th block which is some feature extracted from the image $\phi(x) \in \mathbb{R}^d$: $$ \Phi(x, y) = \left( \overbrace{\underbrace{\mathbf{0}; \mathbf{0}; \dots}_{y-1 \text{ blocks}}; \phi(x); \mathbf{0}; \dots}^{K \text{ blocks}} \right) \in \mathbb{R}^{Kd}. \tag{2} $$ Also, $\Delta(y, y_i) = \mathbf{1}\{y \neq y_i\}$ is the $0/1$ loss function, and the second term in $(1)$ computes the hinge loss of the model on the training example $(x_i, y_i)$.
