I was reading this paper which discusses nonconvex penalized regression. They use the following notation:
- $X\in\mathbb{R}^{n\times p}$ is a data matrix with $n$ observations in $p$ variables
- $\beta\in\mathbb{R}^p$ is a vector of regression parameters
- $\eta=X\beta$ is the expected value of the output $y$
For logistic regression, the goal is to minimize the loss function (eq. (3.2)) $$ Q_{\lambda,\gamma}(\beta)=-\frac{1}{n}\sum_{i=1}^n\{y_i\log\pi_i+(1-y_i)\log(1-\pi_i)\} + \sum_{j=1}^p p_{\lambda,\gamma}(|\beta_j|), $$ where:
- $p_{\lambda,\gamma}$ is a regularizer/penalty function controlled by parameters $\lambda$ and $\gamma$,
- $\pi_i=e^{\eta_i}/(1+e^{\eta_i})$.
On page 13, subsection 4.2 it is written about $Q_{\lambda,\gamma}(\beta)$:
Local convexity diagnostics. However, it is not always necessary to attain global convexity. In high-dimensional settings where $p>n$, global convexity is neither possible nor relevant.
In other words, the number of predictors $p$ is bigger than the number of observations $n$. Why is global convexity not possible in higher dimensions?