1
$\begingroup$

I am looking at the gradient descent method for group lasso questions. Here's what I am currently stuck at.

Given the quadratic form of the objective function: $$ f(x) = \frac{1}{2} x^T V x - m^T x + \sum_{l=1}^g \|x_{G_l}\|_2 $$ where $x$ and $m$ are vectors and $V$ is the positive semidefinite matrix. The $x_{G_l}$ is just referring to the vector of the elements in group $G_l$. The objective function is assumed to be differentiable and convex, and the $\|X_{G_l}\|$ can be $=0$ and $\neq 0$.

I was able to find the partial derivative for the first two components of the objective function: $$ f'(x) = Vx - m $$

However, I became confused with doing partial differentiation for group lasso. I know if I have $\|x\|_2^2$, I can easily differentiate it to $2x$.

Can anyone give an insight on how I should proceed?

Improve this question
$\endgroup$
7
  • $\begingroup$ Is $x\mapsto \|x\|_2$ differentiable for all $x$? $\endgroup$
    – user551504
    Commented Mar 7, 2022 at 3:32
  • $\begingroup$ @user551504 I have added another criterion for the group lasso. $\endgroup$
    – Kevin Choon Liang Yew
    Commented Mar 7, 2022 at 6:02
  • $\begingroup$ The purpose of the group lasso is to induce sparsity, so $x \mapsto \|x\|_2$ is being used specifically because it is not differentiable at $x=0$. So what should we do? One way forward to use subgradient descent instead of gradient descent. $\endgroup$
    – user551504
    Commented Mar 7, 2022 at 15:11
  • $\begingroup$ This is a good way to try at. I'll check about it and see what I can get. Thanks! $\endgroup$
    – Kevin Choon Liang Yew
    Commented Mar 7, 2022 at 16:22
  • 1
    $\begingroup$ OK, the pdf here explains subgradients and gives your problem as an example seas.ucla.edu/~vandenbe/236C/lectures/subgradients.pdf see page 8 in particular $\endgroup$
    – user551504
    Commented Mar 7, 2022 at 17:36

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.