You can train a classifier (e.g. multi-class SVM) with sparsity inducing regularization including $\ell_1$ or $\ell_1/\ell_2$ to ignore dimensions (or groups of dimensions) in the original feature vector that carry little information, as far as the classification task is concerned.

Let $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^n$ denote a set of $n$ labeled training examples where $y_i \in \{1, \dots, K\}$. The training objective of an $\ell_1/\ell_2$ regularized max-margin classifier with hinge loss (multi-class SVM) can be defined as follows: $$ O(w) = \lambda \sum_{g \in G} \sqrt{||w_g||^2} + \sum_{i=1}^n \left( \max_{y=1}^K \left( w \cdot \Phi(x_y, y) + \Delta(y, y_i) \right) - w \cdot \Phi(x_i, y_i) \right) \tag{1} $$ where $w = (w_1; w_2; \dots; w_K)$ is the classifier and $\Phi(x, y)$ is a $Kd$-dimensional feature vector with $K$ blocks of length $d$ each, where all the blocks are zero except for the $y$-th block which is some feature extracted from the image $\phi(x) \in \mathbb{R}^d$: $$ \Phi(x, y) = \left( \overbrace{\underbrace{\mathbf{0}; \mathbf{0}; \dots}_{y-1 \text{ blocks}}; \phi(x); \mathbf{0}; \dots}^{K \text{ blocks}} \right) \in \mathbb{R}^{Kd}. \tag{2} $$ Also, $\Delta(y, y_i) = \mathbf{1}\{y \neq y_i\}$ is the $0/1$ loss function, and the second term in $(1)$ computes the hinge loss of the model on the training example $(x_i, y_i)$.

You can find more details on group sparsity regularization here.

You can train a classifier (e.g. multi-class SVM) with sparsity inducing regularization including $\ell_1$ or $\ell_1/\ell_2$ to ignore dimensions (or groups of dimensions) in the original feature vector that carry little information, as far as the classification task is concerned.

Let $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^n$ denote a set of $n$ labeled training examples where $y_i \in \{1, \dots, K\}$. The training objective of an $\ell_1/\ell_2$ regularized max-margin classifier with hinge loss (multi-class SVM) can be defined as follows: $$ O(w) = \lambda \sum_{g \in G} \sqrt{||w_g||^2} + \sum_{i=1}^n \left( \max_{y=1}^K \left( w \cdot \Phi(x_y, y) + \Delta(y, y_i) \right) - w \cdot \Phi(x_i, y_i) \right) \tag{1} $$ where $w = (w_1; w_2; \dots; w_K)$ is the classifier and $\Phi(x, y)$ is a $Kd$-dimensional feature vector with $K$ blocks of length $d$ each, where all the blocks are zero except for the $y$-th block which is some feature extracted from the image $\phi(x) \in \mathbb{R}^d$: $$ \Phi(x, y) = \left( \overbrace{\underbrace{\mathbf{0}; \mathbf{0}; \dots}_{y-1 \text{ blocks}}; \phi(x); \mathbf{0}; \dots}^{K \text{ blocks}} \right) \in \mathbb{R}^{Kd}. \tag{2} $$ Also, $\Delta(y, y_i) = \mathbf{1}\{y \neq y_i\}$ is the $0/1$ loss function, and the second term in $(1)$ computes the hinge loss of the model on the training example $(x_i, y_i)$.

References

• You can find more details on group sparsity regularization here.

• implementation of $\ell_1$ regularized multi-class SVM implementation in C++ with Python and Matlab interface can be found here (public source code).

You can train a classifier (e.g. multi-class SVM) with sparsity inducing regularization including $\ell_1$ or $\ell_1/\ell_2$ to ignore dimensions (or groups of dimensions) in the original feature vector that carry little information, as far as the classification task is concerned.

Let $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^n$ denote a set of $n$ labeled training examples where $y_i \in \{1, \dots, K\}$. The training objective of an $\ell_1/\ell_2$ regularized max-margin classifier with hinge loss (e.g. multi-class SVM) can be defined as follows: $$ O(w) = \lambda \sum_{g \in G} \sqrt{||w_g||^2} + \sum_{i=1}^n \left( \max_{y=1}^K \left( w \cdot \Phi(x_y, y) + \Delta(y, y_i) \right) - w \cdot \Phi(x_i, y_i) \right) \tag{1} $$ where $w = (w_1; w_2; \dots; w_K)$ is the classifier and $\Phi(x, y)$ is a $Kd$-dimensional feature vector with $K$ blocks of length $d$ each, where all the blocks are zero except for the $y$-th block which is some feature extracted from the image $\phi(x) \in \mathbb{R}^d$: $$ \Phi(x, y) = \left( \overbrace{\underbrace{\mathbf{0}; \mathbf{0}; \dots}_{y-1 \text{ blocks}}; \phi(x); \mathbf{0}; \dots}^{K \text{ blocks}} \right) \in \mathbb{R}^{Kd}. \tag{2} $$ Also, $\Delta(y, y_i) = \mathbf{1}\{y \neq y_i\}$ is the $0/1$ loss function, and the second term in $(1)$ computes the hinge loss of the model on the training example $(x_i, y_i)$.

You can find more details on group sparsity regularization here.

You can train a classifier (e.g. multi-class SVM) with sparsity inducing regularization including $\ell_1$ or $\ell_1/\ell_2$ to ignore dimensions (or groups of dimensions) in the original feature vector that carry little information, as far as the classification task is concerned.

Let $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^n$ denote a set of $n$ labeled training examples where $y_i \in \{1, \dots, K\}$. The training objective of an $\ell_1/\ell_2$ regularized max-margin classifier with hinge loss (multi-class SVM) can be defined as follows: $$ O(w) = \lambda \sum_{g \in G} \sqrt{||w_g||^2} + \sum_{i=1}^n \left( \max_{y=1}^K \left( w \cdot \Phi(x_y, y) + \Delta(y, y_i) \right) - w \cdot \Phi(x_i, y_i) \right) \tag{1} $$ where $w = (w_1; w_2; \dots; w_K)$ is the classifier and $\Phi(x, y)$ is a $Kd$-dimensional feature vector with $K$ blocks of length $d$ each, where all the blocks are zero except for the $y$-th block which is some feature extracted from the image $\phi(x) \in \mathbb{R}^d$: $$ \Phi(x, y) = \left( \overbrace{\underbrace{\mathbf{0}; \mathbf{0}; \dots}_{y-1 \text{ blocks}}; \phi(x); \mathbf{0}; \dots}^{K \text{ blocks}} \right) \in \mathbb{R}^{Kd}. \tag{2} $$ Also, $\Delta(y, y_i) = \mathbf{1}\{y \neq y_i\}$ is the $0/1$ loss function, and the second term in $(1)$ computes the hinge loss of the model on the training example $(x_i, y_i)$.

You can find more details on group sparsity regularization here.