how to use gradient descent to solve ridge regression with a positivity constraint? I am working on ridge regression at this moment:
$$\sum{(y_{i}-\beta\cdot X_{i})}^{2} + \lambda \cdot\beta^{2}$$
with an additional constraint:

for some $\beta_{m} > 0 $ and some $\beta_{n}<0$, while $n+m$ is the total number of predictors.

Basically, there is a positivity constraint here.
The positivity constraint makes the target function non-differentiable. I am not sure if this can still be solved by a gradient descent approach.
Can anyone share any insight here?
 A: Two recommendations depending on which case you're in. To give some immediate context, Ridge Regression (aka Tikhonov regularization) solves the following quadratic optimization problem:
$$
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $\mathbf{b}$)} & \sum_i \left(y_i - \mathbf{x}_i \cdot \mathbf{b} \right)^2 + \lambda \|\mathbf{b}\|^2_2  \\
 \end{array}
$$
This is ordinary least squares plus a penalty proportional to the square of the $L_2$ norm of $\mathbf{b}$.
You want to add the linear constraints that $b_j \geq 0$ for $j \in J$ and $b_k \leq 0$ for $k \in K$.
Case 1: self-study, if you're trying to learn how things work
One possible approach is to add a barrier function to your objective function for each constraint. Then run gradient descent etc... on your new objective function. This is known as an interior point method.
For example, instead of the objective:
$$
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $b$)} & f(b)  \\
 \mbox{subject to} & b \geq 0
 \end{array}
$$
You could have:
$$
 \begin{array}{*2{>{\displaystyle}r}}
 \mbox{minimize (over $b$)} & f(b) - \frac{1}{t}\log(b)
 \end{array}
$$
Where $t$ (for $t > 0$) is some parameter that controls how sharp your barrier/penalty is. As $t$ becomes larger, the two problems become equivalent.
See the section on barrier functions and interior point methods in Boyd's Convex Optimization book.
For example, here's a graph of $y = - \frac{1}{20} \log x$. As $x$ approaches $0$, the penalty $- \frac{1}{20} \log x$ approaches $\infty$.

Case 2: If you just want a numerical solution
This is a quadratic optimization problem with linear constraints. There are numerous extremely well tested, optimized libraries/engines to solve this type of problem. In Matlab, one would invoke the quadprog function.
