SVM with non-negative weights

Question

An SVM classifier can be obtained by solving the following,

$\arg\min \frac{1}{2}\|W\|_2^2 + C\sum_i \max(0, 1-y_i (W^T\mathbf{x}_i + b))$

where $W$ is the hyperplane (or weights), $b$ is the bias, $y_i$ is the label and $\mathbf{x}_i$ is the feature of an instance $i$.

For some reason I need to constaint all the elements in $W$ should be non-negative, i.e. $w_j \geq 0, \forall j$. Bias could take any values.

1. Is this reasonable?
2. If it is, are there any pakages (e.g. liblinear) I can use directly for this?
3. How can I optimize $W$ with non-negative constraints?

Answer 1

Initially, I think the model is fine but I don't know any package can solve this model directly.

One method is that you may can try to write the dual form of this problem. I believe it will be a QP problem with extra nonnegative constrains.

Alternatively, you also can try to find some algorithms which updates $w_{i}$ separately so that you can slightly rewrite the iteration formulas with $w_i^* = max(w_i, 0)$.

Answer 2

As @Ben Dai suggests, this looks like it will be a quadratic programming problem with inequality constraints. There are various general quadratic programming packages available which may be used to solve the problem, such as the quadprog routine in the MATLAB optimisation toolbox. This is only such routine I have used, but there are many other packages for this sort of problem.