My dataset have 141 variables, all are numeric. To do clustering based on them, it seems that PCA is required to reduce dimensionality.

The var plot shows that variance among these variables are unstable. ( variance of all variables are under 0.1)

I scaled the dataset, and the var plot shows stability.

enter image description here

Then I do PCA on it and make a scree plot to show the percentage of variance in each components. I was shock that first 10 components only takes up 20% of the total variance (The chosen factors should explain 70 to 80% of variance at least).

I tried PCA without scaling, and still find that first 10 components explain less than 50% of variance.

scree plot of variance percentage before scaling

Does it mean that the PCA is not required, even if the number of variables are large? And do I need to scale the data?

  • $\begingroup$ On your scree plot, 4 first PCs explain roughly 52% of the total variance. That is not bad. $\endgroup$
    – ttnphns
    Jul 27 '19 at 18:56

There is no rule that says "thou must scale your data with PCA".

There are plenty of cases where scaling is bad, for example with you have latitude and longitude.

Scaling is often better than not scaling if the axes are very different. PCA is often helpful if you have linear correlations and want to get rid of this redundancy in the data. If you want to retain this redundancy (because it helps solving your problem), PCA can even be harmful!

So whenever you have doubt, don't blindly apply any such transformation, but rather try to understand your problem better. What the right approach is depends on your data and objective, not on some textbook heuristic such as using PCA.

In your plot it does appear as if you have some near-duplicate variables. You may want to first merge these pairs. Then check for further highly correlated variables. Because these harm PCA, usually the result gets much more meaningful when you have eliminated the obvious relationships. Also get rid of any clearly useless variables.

  • $\begingroup$ Thanks! What do you mean by "Scaling is often better than not scaling if the axes are very different"? $\endgroup$
    – Frida Guo
    Jul 27 '19 at 21:00
  • $\begingroup$ My data is a large dataset recording online shoppers' behaviors. It has 141 variables, first 6 variables are time orientation, named like "weekday" and "weekend", and the values are percentage between 0 and 1, which indicates the percentage this user shopped in weekday or weekend. The variable "weekday" and "weekend" have correlation coefficient at -1, so I exclude the "weekend" to deal with "further highly correlated variables" as you suggested. $\endgroup$
    – Frida Guo
    Jul 27 '19 at 21:04
  • $\begingroup$ And the rest 130+ variables are products' name, with the values indicate that how many times the customer ordered this product. Many values are 0, and most of values are integer less than 5. I want to keep all these variables for clustering, and these variables have very low correlation (after I exclude "weekend"). In this case do you suggest me to use PCA or not? Can you recommend some literatures about that? $\endgroup$
    – Frida Guo
    Jul 27 '19 at 21:14
  • $\begingroup$ Sounds like PCA should at most be applied to the remaining weekday encoding. But because weekday and products are stacked into the same data frame I am sceptical about the results that can be obtained this way. You're mixing input and output variables, for example. $\endgroup$ Jul 27 '19 at 23:46
  • $\begingroup$ What do you mean by "mixing input and output variables"? $\endgroup$
    – Frida Guo
    Jul 28 '19 at 1:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.