What to conclude for the data-set when the variance for principal components is too low or too high?

Question

I am working on analysing and visualizing a dataset having 12 features and came across PCA. I reduced the dataset to 2 principal components which together explain a variance of 18%. I was able to plot the data and see if there is a tendency to form natural clusters within the data. But I could not understand what to conclude where the first two principal components only explain 18% of the variance in the whole data.

So, what could be concluded in general when:

• The variance suggested by the first two principal components is very high as being close 90%
• The variance suggested by the first two principal components is very low as 18% as has happened in my case

Thinking about the above 2 questions also makes me think about the assumptions PCA makes about data? What data would not be a good fit for PCA / or when should I avoid using PCA?

Edit:

Following is the loadings table, which I computed using python's scikit as :

pca.components_.T * np.sqrt(pca.explained_variance_)

Loadings:

[[ 0.8616755  -0.15340052]
[-0.42014373  0.38166004]
[ 0.81644887 -0.21071762]
[ 0.25729266  0.37770332]
[ 0.37376327  0.205526  ]
[-0.0636729   0.71293634]
[ 0.04151499  0.79056501]
[ 0.69621113  0.32425081]
[-0.77222693  0.00931596]
[ 0.42778107 -0.05213256]
[-0.19940009 -0.53609858]]

Also, here are the first 2 principal components created from the dataset:

[[ 0.48931422 -0.23858436  0.46363166  0.14610715  0.21224658 -0.03615752
0.02357485  0.39535301 -0.43851962  0.24292133 -0.11323207]

[-0.11050274  0.27493048 -0.15179136  0.27208024  0.14805156  0.51356681
0.56948696  0.23357549  0.00671079 -0.03755392 -0.38618096]]

Answer 1

Think about the extreme cases:

• If your 12 features have no information in common, you would need 12 components to capture 100% of the information (each component just contains the information of one feature, and PCA is useless).

• If your 12 features have everything in common, you can fit 100% of the information into 1 component (that one component will be identical to the 12 identical features, so PCA is useless).

Real-world data is usually somewhere in between, and a decision on whether PCA is worthwhile is usually based on context (what are the features?) and the loadings of each feature.

You were wondering why the two most important components only account for 18% of the variance. The easy, general answer is that this group of 12 features don't seem to have a lot in common.

A more nuanced explanation comes from looking at the loadings...

...and noticing that:

• Some features load relatively high on the first component but low on the second one (blue).
• Some features load very low on the first component and relatively high on the second one (red).
• Other features, like green, have relatively important proportions of their variance in both components.

So, even though the group of features is not very homogeneous, we can see that some of them are more related to each other than others.