Training an SVM classifier with non-negative weight constraint I have a problem, where I need to learn a classifier (such as SVM) such that all the learned weights to be non-negative due a constraint on the classifier function. 
I found out that "SVM Struct" is able to do as mentioned in the comments inside some of their source codes. But, I'm still unable to find any reference (paper, or lecture notes) describing the theoretical aspects of how to solve this problem. Is there any standard method for integrating such constraints in the classification weight learning process? Note that a similar question has been asked in the following question, but I believe no clear and complete answer is given to it. 
 A: I successfully used a non-negative weights linear Support Vector Machine (SVM) by adding some dummy data (all possible one-hot-vector excluding the bias) at training time and using off-the-shelf SVM implementations such as LibLinear but any implementation could do -- see my blog post. My goal was to fastly achieve diagonal Mahalanobis metric learning. Be careful with the class weighting (wi in LibLinear).
A: I don't have any references on this either, but maybe the following thoughts will be helpful. First, why would you need all $w_i$ to be non-negative? If you need a black box that classifies your data, why would you care about how the box works? If you are studying your data, the negativity of $w_i$ is a property of the data reflected in the SVM classifier, then why deny it?
Answering your question, I'd suggest the following two approaches.


*

*Add constraints $w_i \geq 0$ to the original SVM optimization problem. I don't know if there are any SVM libraries that would allow you to do that. If your dataset is not very large, you can solve this new optimization problem yourself using any general solver for quadratic problems. Note that, generally speaking, the properties of your hyperplane will not be the same as those of the common SVM hyperplane.

*Get $w, b$ by solving the original SVM optimization problem and if some $w_i$ are negative, transform your data using one of the following approaches.


*

*For each $w_i < 0$ you can consider $x'_i = -1 \times x_i$ ("reflection" of the $i$-th dimension).

*Shift and rotate your data to get all $w_i \geq 0$. First, substract $b/||w||$ from your data -- now your hyperplane passes through the origin. Rotate your data to point $w$ into the positive quadrant of $R^n$. For example, if $||w|| = d$, you can calculate the angle $\alpha$ between $w$ and $w' = (d, 0, ..., 0)$, and then rotate your dataset by $\alpha$. Note that in this case you will get a classifier with all $w'_i = 0$ except $w'_1 = d$.


A: The decision rule for a (linear kernel) SVM is given by $f(x_{new}) = \textrm{sign}(\sum_{i}\alpha_{i}x_{i}^{T}x_{new} + b)$, where $\alpha_{i}$ is non-zero for only a subset of the $x_{i}$ (the support vectors) and I lump the training label $y_{i}$ in with $\alpha_{i}$ as part of the "weight" for support vector $x_{i}$.
The decision rule is generally never expressed as $f(x_{new}) = \textrm{sign}(w^{T}x_{new} + b)$, with $(w,b)$ being the parameters for the optimal separating hyperplane from the SVM optimization. 
If you're looking for algorithms where representing the decision rule in that particular inner-product format is desirable, SVM may not be appropriate for you. You could instead use constrained logistic regression if you want to enforce the hyperplane coefficient condition $w_{i} > 0, \forall{} i$, but you make the optimization much harder and lose guarantees on properties of the classifier.
I haven't thought too much about it, but you may also lose feasibility. At the very least, you must by definition produce a classifier that has a worse margin than the traditional SVM. Suppose you were only considering a linear kernel. SVM will pick out the line (among all lines) that yields maximum margin in the data (in the case it is linearly separable). If you add a constraint like $w_{i} > 0, \forall{} i$, then you are asking the SVM algorithm to search only over a special subset of lines, with positive coefficients. So the best achievable margin there will be no better than the best margin among all possible lines, and probably a lot worse. And you will lose the ability to use the same Lagrangian dual representation that allows for the support-vector based decision rule I mentioned above, since the dual will now involve new constraints.
Can you share more information about the nature of your problem and why it requires this constraint?
Lastly, as far as I can tell, the mention of SVM-struct is entirely non-sequitur. That is a method for having vector- or structure-valued outputs instead of merely having class labels (some scalar enumeration data type) as the outputs.
A: I developed a Python library to solve the linear SVM with nonnegative constraint, see https://github.com/statmlben/Variant-SVM.
