# Group Lasso optimization

I read that, for the group lasso, to solve the zero subgradient equations, one approach involves keeping all block vectors fixed, denoted as $$\{\hat\theta_k, k \ne j\}$$, and then solving for $$\hat \theta_j$$. This procedure corresponds to employing block coordinate descent on the group lasso objective function. Given the convex nature of the problem and the block-separable penalty, convergence to an optimal solution is guaranteed (Tseng 1993). With all $$\{\hat\theta_k, k \ne j\}$$ held constant, the equation becomes:

$$$$-\mathrm{\it{Z}}_{j}^{T}(r_{j}-\mathrm{\it{Z}}_{j}\widehat{\theta}_{j})+\lambda\widehat{s}_{j}=0,$$$$

From the conditions satisfied by the subgradient $$\hat s_j$$, that is the subgradient of the euclidean norm, we must have $$\hat \theta = 0$$ if $$||Z_j^Tr_j||_2 < \lambda$$, and otherwise the minimizer $$\hat \theta_j$$ must satisfy:

$$$$\widehat{\theta}_{j}=\left(\mathbf{Z}_{j}^{T}\mathbf{Z}_{j}+{\frac{\lambda}{||\widehat{\theta}_{j}||_{2}}} I \right)^{-1}\mathbf{Z}_{j}^{T}r_{j}.$$$$

It's important to note that the above equation lacks a closed-form solution for $$\hat \theta_j$$ unless $$Z_j$$ is orthonormal. IN THis special case, we have the simple update:

$$$$\widehat{\theta}_{j}=({1-\frac{\lambda}{||Z_j^Tr_j||_2})_{+} Z_j^Tr_j}$$$$

where $$(t)_+ := max\{0,t\}$$ is the positive part of the function. I do not understand where does this last formula come from. What happens when we have orthonormality? How do we arrive to define equation 3?