# Can I penalize an arbitrary regression model and get Elastic-Net-esque results?

Consider an arbitrary-ish regression model with the unpenalized likelihood $$\log \mathcal{L} = \sum_i f\left(y_i\,|\,g(\beta_0 + \beta x_i)\right)$$

with $\beta = \left(\beta_1, \dots, \beta_K\right)$ and $x_i$ being a data vector.

If I instead maximize $$\log \mathcal{L} = \sum_i f\left(y_i\,|\,g(\beta_0 + \beta x_i)\right) + \lambda P_\alpha(\beta)$$ where $P_\alpha(\beta) = \sum_{k=1}^K \frac{1}{2}(1-\alpha)\beta_k^2 + \alpha | \beta_k |$, will my estimated $\beta$ have the same nice properties for an arbitrary convex $f$ as for a well-studied $f$ like the Bernoulli or Gaussian likelihoods studied in Friedman, Hastie, and Tibshirani (2010)?

Specifically, what if $y$ is censored? That is, $y = \cases{y^* &\text{if}\ y^* < c \\ c &\text{if}\ y^* \geq c}$. Then my likelihood is something like $$\log \mathcal{L} = \sum_{i\ : y^*_i < c} F^*\left(y^*_i\,|\,g(\beta_0 + \beta x_i)\right)\,f^*\left(y^*_i\,|\,g(\beta_0 + \beta x_i)\right) \\+ \sum_{i\ : y^*_i \geq c} 1 - F^*(c\,|\,g(\beta_0 + \beta x_i)) \\+ \lambda P_\alpha(\beta)$$ Is there any reason why this wouldn't work?

I imagine this is related to the question Do I get the nice asymptotic properties of MLE when I restrict the parameter space?

• "the addition of convex penalties usually preserves convexity" – it always does, the sum of convex functions is convex, and the question specified a convex $f$. I think the question is about whether the solution will be sparse and reasonably regularized in a "nice" way, not whether the optimization problem remains tractable. – Dougal Apr 21 '15 at 23:56