Can someone help me on Sparse PCA? I am using the "elasticnet" package to perform sparse PCA. I am having a hard time in figuring out how many nonzero values should a component contain?

For example, In this code:

sparse.pca.result  <-  spca(X, K = 2, type = "predictor", sparse = "varnum", para = c(4, 4))

(para = c(4, 4)) indicates the number of non-zero components for each of the two PC’s respectively.

So the question is, how to identify the number of non-zero components?

I hope that someone could help me on this.

  • $\begingroup$ Also, can someone please give me good sites or references about sparse PCA? Preferrably with examples and interpretations $\endgroup$ – Jonalyn Jul 27 '16 at 15:16
  • $\begingroup$ The reference is web.stanford.edu/~hastie/Papers/spc_jcgs.pdf , there are examples in section 5. $\endgroup$ – altroware Mar 24 '20 at 8:48

One way to do this a little more adaptively is to specify the penalty instead of the number of nonzeroes you want. (Set sparse='penalty'.) Then you can specify the same parameter value across all components, while still allowing them to have different levels of sparsity.

Ordinarily, one might select the penalty parameter via cross-validation, but spca cannot deal with missing values, so it's hard to hold out a set of entries. You could hack together something similar, though:

  • Train the SPCA, holding out one case. You'll obtain L and R such that $X_{-1} \approx LR$. (Suppose $X_{-1}$ has cases as rows, and you left out row 1.)
  • Using half the variables (columns), fit a regression model to predict $X_{1}$ from $R$. You'll obtain $L_1$ such that $L_1R \approx X_{1}$.
  • Measure the predictive accuracy of the regression model using the other half of the variables.

You should probably hold out more than 1 row at a time.

I'll look for references if anybody's still looking at this page -- drop me a comment.

  • $\begingroup$ I'm looking at this pages, and yes, can you please dig more references for this? $\endgroup$ – altroware Mar 24 '20 at 8:50
  • 1
    $\begingroup$ It seems to be standard practice in collaborative filtering, but I haven't found academic papers after a light search. This is also tricky and I think I may have screwed it up. I'll give it some thought and try to get back to you soon. $\endgroup$ – eric_kernfeld Mar 25 '20 at 21:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.