-1
$\begingroup$

as I have only learned PCA for a short while, some problems occured when I faced practice. I am willing to accept solution or advice for the following content and great many thanks for anyone's kindly help.

Edit: the dataset will be used in a prediction model.

Supposed that there is one dataset with 300 characteristics and they are grouped by two different collecting methods, each of which has the same 150 characteristics. As these 2 collecting methods have much in common, the observations between them should be similar. Correlation matrix of the whole dataset lies below,

enter image description here

The lower left part (part 1) of the matrix can be viewed as the correlation matrix of the characteristics within method 1, and the upper left part (part 2) of the matrix describes the correlation of the characteristics across the two methods. Here's my first question: what can we infer from the similar pattern between part 1 and part 2 in light of dimensionality reduction? Is it possible to cut the number of characteristics by half with the information given in this graph?

Moving on to the next stage, I performed PCA to the whole dataset. The result showed that in order to obtain 90% of the information of original dataset, I needed to keep 67 dimensions. Screeplot and biplot of the result lie below, Screeplot

Biplot And my second question is just like the title of this question: is it acceptable to reduce dimensions from over 300 to 67? Since 67 is still a great number, I don't quite know whether it is useful or not to perform such a PCA. Or might there be any way of enhancing the effect?

My third question is that: what can one infer from the pattern of this biplot with first two dimensions?

Thanks so much for reading my questions, and they lie below,

  1. What can we infer from the similar pattern between part 1 and part 2 in light of dimensionality reduction? Is it possible to cut the number of characteristics by half with the information given in this graph?

  2. Is it acceptable to reduce dimensions from over 300 to 67? Since 67 is still a great number, I don't quite know whether it is useful or not to perform such a PCA. Or might there be any way of enhancing the effect?

  3. What can one infer from the pattern of this biplot with first two dimensions?

$\endgroup$
  • $\begingroup$ I quote your second question “Is it acceptable to reduce dimensions from over 300 to 67? Since 67 is still a great number, I don't quite know whether it is useful or not to perform such a PCA. Or might there be any way of enhancing the effect?” 67 is far less than 300, for a loss of 10% of the total variability/into.. it is clearly a far lower number of coefficients to be estimated. Which means usually faster resolution of the problem and less parameters in general (taking the most representative features). It is a good reduction.. $\endgroup$ – Fr1 Aug 8 at 3:20
  • 1
    $\begingroup$ What's missing from this question is a goal. Why perform PCA at all? Do you intend to use the remaining PCs as variables in a prediction model? Have you encountered problems just using all 300 variables? The part about grouping by collection methods also isn't clear to me. Are you saying there is dependence in the data? $\endgroup$ – Frans Rodenburg Aug 8 at 3:32
  • $\begingroup$ @FransRodenburg, thanks for suggestion. Indeed I'd like to use PCs in a prediction model. As for the reason why I needed to reduce the dimension, I suppose that there might be correlation between variables in the same group or across the group, so 300 variables may be redundant. $\endgroup$ – C. Augustus Aug 8 at 3:40
  • $\begingroup$ @Fr1, thank you. As I may face dataset with more variables, say, 1500, I asked this question for finding a robust way to process those variables. What I truly wonder is that whether a process simply consisted of performing PCA, finding acceptable dimensions, deploying them to create new features is robust or not. Probably I have not obtained adequate comprehension of PCA and dimensionality reduction. $\endgroup$ – C. Augustus Aug 8 at 3:53
2
$\begingroup$

There are several issues here.

First, can you get rid of half the variables at once by either using only one measurement method or by taking the average? The upper left part of your first plot is not ideal for assessing this, because, for this question, we only care about the correlation across methods for particular variables. But it looks like all the points on the 45 degree line are very red, indicating very high correlation. I'd examine them more closely first, but it looks like the answer is yes. So, you are down to 150 variables.

Second, can you reduce to 67 from 300? Well, sure. Why not? But why are you set on 90% of the variance? The first 5 components account for about 80% of the variance (34+18+10+10+8). That seems like a good place to stop.

Third, if you are using this for prediction, you might consider partial least squares instead of PCA.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.