1
$\begingroup$

I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper.

To derive the sparse axis, they use the sparse PCA criterion $$(\hat{A}, \hat{B}) = \operatorname{arg min}_{A, B} \sum_{i=1}^{n} \left|\left|x_i - AB^Tx_i\right|\right|_2^2 + \lambda_2 \sum_{j=1}^{k}\left|\left|\beta_j\right|\right|_2^2 + \sum_{j=1}^k \lambda_{1,j} \left|\left|\beta_j\right|\right|_1$$ $$\text{subject to } A^TA = I_{k \times k}$$

where $\lambda_{i,j}$ is the sparsity controlling parameter. Solving this criterion by alternating minimization confronts us with an elastic net problem which is computationally hard in the $p \gg n$ case. (Does anyone have a complexity bound for the coordinate descent approach?)

In their paper, they use the fact that if we choose $\lambda_2 \to \infty$ there exists an explicit solution to the elastic net problem. However, why would we choose $\lambda_2 \to \infty$? In what sense does this still solve the original problem? Doesn't this just result in all my axis being 0 since I increase the penalty to infinity?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.