To select the important variables (features) in your dataset, you will need more than the output you have described. These output are useful for selection procedures for finding the number of components, but not the necessary variables. S corresponds to singular values (the square root of the eigenvalues), and LATENT to eigenvalues. Thus, they give a sense of the amount of variation captured by the dimensions of the transformed variables.
A more useful output are the eigenvectors. These are V in the SVD output. These are the COEFF output from PCA. The elements of the eigenvectors are the coefficients in the linear relationships defined by PCA. Thus a variable that has very low values for all of the retained eigenvectors is potentially excludeable. The simplest way to exclude unnecessary variables is to set a threshold: if the variable has coefficients that are all below a threshold, it is dropped. This is called simple thresholdling. However, Cadima and Jolliﬀe (1995) show that the results can be misleading
- Cadima, J. and Jolliffe, I. T. (1995). Loadings and Correlations in the Interpretation of Principal Components. Journal of Applied Statistics, 22(2), 203–214.
A better approach would be to try a sparse PCA method. Two are: SCoTLASS, detailed in the paper below:
- Jolliﬀe, I. T., Trendaﬁlov, N. T., and Uddin, M. (2003). A Modiﬁed Principal Component Technique Based on the LASSO. Journal of Computational and Graphical Statistics, 12(3), 531–547.
and SPCA, introduced in the following paper:
- Zou, H., Hastie, T., and Tibshirani, R. (2006). Sparse Principal Component Analysis. Journal of Computational and Graphical Statistics, 15(2), 265–286.
SPCA is implemented in a MATLAB toolbox. I would recommend trying spca() in the Matlab toolbox with the "soft thresholding" option to see if it helps (see example in the spca code).
If you do not use soft thresholding (an automatic, data driven sparsity parameter selection procedure) you will have to select the sparsity parameters yourself. Consult the SCoTLASS paper or the SPCA paper for guidelines on parameter selection. You will want to select a sparsity parameter that excludes unnecessary variables, but is not so strict that important ones are also dropped.
After you have performed a sparse PCA procedure, the unnecessary variables should hopefully have very low coefficient values (typically low enough that you can safely apply an epsilon small threshold without encountering the problems highlighted in Cadima '95) and you will be able to clearly identify variables that are important in your data.