Is logstic regression with elasticnet penalty a convex optimization problem? Logistic regression is a convex optimization problem and adding elastic net penalties is adding convex elements. So am I right if I conclude that logistic regression with elastic net penalties is also convex, and does this only hold in the special cases of the L1 and L2 penalties?
 A: Since the loss used in logistic regression, the $\ell_1$, and the $\ell_2$ norm are convex, any non-negative combination of those terms defines a convex problem. Since $0$ is technically non-negative, this also means you can remove any of the terms from the objective and still have a convex problem.
Here's Why.
Suppose that $\{f_1,\dots,f_n\}$ are a set of real-valued convex functions. Then, 
$$f(x) = \sum_{i=1}^n \lambda_i f_i(x)$$
is convex in $x\in\mathbb{R}^n$ for scalars $\lambda_i \geq 0$. You can see why this is the case simply by applying Jensen's inequality. Specifically, we know that for any $x$ and $y$,
$$\lambda_i f_i\left(\theta x + (1-\theta)y\right) \leq \theta \lambda_i f_i(x) + (1-\theta) \lambda_i f_i(y)$$
for all $i \in \{1,\dots,n\}$, where $\theta \in (0,1)$. Substituting this into the sum, we have
\begin{align}
f\left(\theta x + (1-\theta)y\right) &= \sum_i \lambda_i f_i\left(\theta x + (1-\theta)y\right) \\
&\leq \sum_i \theta\lambda_i f_i\left(x\right) + (1-\theta)\lambda_i f\left(y\right) \\
&= \theta f(x) + (1-\theta) f(y)
\end{align}
meaning that $f$ satisfies Jensen's inequality and is therefore convex. This holds for the special case Mark mentioned in the comment, for which $\lambda_i = 1$ for all $i$.
