I'm currently in the process of trying to understand the paper Regularization Paths for Generalized Linear Models via Coordinate Descent by Friedman et al. with regard to the regularization of logistic regression. Unfortunately, I was not able to figure out the exact algorithm that is used for optimization. I was looking for other papers that use coordinate descent in this context and came across Sparse logistic regression for text categorization by Genkin et al. They go into more detail and in section 3.1 they mention
Let $f(\beta_j)$ be the objective expressed as the function of only one component $\beta_j$ with all the rest being fixed, and $Q(\beta_j, \Delta_j)$ be an upper bound on the second derivative of $f (\cdot)$ in the $\Delta_j$-vicinity of $\beta_j$ , $\Delta_j > 0$:
$Q(\beta_j , \Delta_j ) \geq f (\beta_j + \delta)$ for all $\delta \in [−\Delta_j , \Delta_j ]$.
Later on they use the following formula as upper bound for L2-penalized logistic regression:
$Q(\beta_j , \Delta_j ) = \sum_i x_{ij}^2 F(\beta^T x_i y_i, \Delta_j x_{ij}) + 2\lambda$
$F(r, \delta) = 0.25$ if $|r| \leq |\delta|$
$F(r, \delta) = 1 / (2 + \exp(|r| - |\delta|) + \exp(|\delta| - |r|))$ otherwise.
I was surprised to see that this apparently necessary upper bound was not mentioned in the paper by Friedman et al. In addition, the bound differs between L1 and L2 penalty. What would the bound look in the generic framework proposed by Friedman et al. which is applicable for a wide range of penalties?
My second question is related to the upper bound proposed by Genkin et al. They did not provide or reference a prove that validates there choice. Am I missing something obvious here?