# Logistic regression subgradients with different regularizers

Consider the following regularized logistic regression problem:

$$\textbf{w}^* = \min_{\textbf{w} \in {\rm I\!R}^p} \mathcal{L}(\textbf{w})$$

where

$$\mathcal{L}(\textbf{w}) = \frac{1}{n} \sum_{i=1:n} \{-y_i \textbf{w}^T \textbf{x}_i + \log(1 + \exp(-\textbf{w}^T \textbf{x}_i)) \} + \lambda G(\textbf{w})$$

1. Derive subgradient for L(w) if $$G(\textbf{w}) = \sum_{i=1:p} (a|w_i|^{\frac{1}{2}} + b)^2$$

2. Derive a subgradient stochastic gradient descent algorithm for when $$G(\textbf{w}) = \sum_{i=1:p}|w_i|$$

3. If $$G(\textbf{w}) = \frac{1}{2} \sum_{i=1:p} w_i^2$$ what is the expected runtimes for Gradient descent and Stochastic Gradient Descent in terms of $$(n, \epsilon)$$

I'm struggling with the calculus for these problems. I know that for the subgradient of the L1 norm we'll end up with 3 cases, but I'm not sure what form the final answer should take.