I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper.

To derive the sparse axis, they use the sparse PCA criterion $$(\hat{A}, \hat{B}) = \operatorname{arg min}_{A, B} \sum_{i=1}^{n} \left|\left|x_i - AB^Tx_i\right|\right|_2^2 + \lambda_2 \sum_{j=1}^{k}\left|\left|\beta_j\right|\right|_2^2 + \sum_{j=1}^k \lambda_{1,j} \left|\left|\beta_j\right|\right|_1$$ $$\text{subject to } A^TA = I_{k \times k}$$

where $\lambda_{i,j}$ is the sparsity controlling parameter. Solving this criterion by alternating minimization confronts us with an elastic net problem which is computationally hard in the $p \gg n$ case. (Does anyone have a complexity bound for the coordinate descent approach?)

In their paper, they use the fact that if we choose $\lambda_2 \to \infty$ there exists an explicit solution to the elastic net problem. However, why would we choose $\lambda_2 \to \infty$? In what sense does this still solve the original problem? Doesn't this just result in all my axis being 0 since I increase the penalty to infinity?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.