I'm new to canonical correlation analysis. I'm running a sparse canonical correlation analysis in R using the PMA package. My first question is why the correlation coefficients associated to the canonical dimensions Cor(Xu,Zv) do not strictly decrease. My second question is whether there is a way to select the most significant canonical dimensions (something like the 'elbow method' in PCA).
When you perform CCA, you are finding correlations across all of your data landscape. It is possible that there is some correlation left over, particularly when using sparse CCA. The sparse part, by forcing most of the betas to zero, may leave some correlation that you may be extracting in the next step. Thus, the next correlation may include some of this, and so not be diminished. This is not the same as PCA, where you extract all of the correlation and maintain all of the betas in one step. In my experience, PMA is the best package thus far, and so I think you are using the right tools.