What does it mean when PCA does not produce a reduction in dimensionality? I am a beginner in PCA, I am trying to apply it on a dataset I have.
The features are different geometrical parameters with different units and variability, I do standardize the features matrix by subtracting the mean and dividing by the standard deviation.
For the PCA, I use the PCA() method in sklearn wich is based on the singular value decomposition (SVD). Once I fit the model I get the following results if I choose just 4 components:
Is this telling me something about my features? or there is something fundamentally wrong in my approach?
Thank you!
 A: PCA can be used to scale and rotate data, if we select all transformed features instead of a subset of them. My answer here gives an example of scaling and rotating the data, but without dimension reduction. How to decide between PCA and logistic regression?

Your plot shows that if we use all 12 features, the variance explained is 100%, i.e., without information loss. But if you select number of features smaller than 12, there will be information loss.
Note that in most cases, PCA can reduce dimension but at the cost of losing information. If you want to keep 99% variance, unless you have highly correlated (redundant features), PCA will not able to help.
In other words, your plot shows there are not too much redundancies in your data set.

Here are examples of both cases (with 5 features).
set.seed(0)
x1=matrix(rnorm(1000),ncol=5)

x2 = matrix(rnorm(600),ncol=3)
x2=cbind(x2,x2[,3]*runif(200)*0.01)
x2=cbind(x2,x2[,3]*runif(200)*0.01)

you may run PCA on x1 and x2 to see the difference on variance explained respect to number of features selected.
You would see for x2, 3 features will explain most of the variance, because other two features are highly correlated with the third column of x2.
A: Results suggest that your features are mutually orthogonal. Accounting for total variance means accounting for both variance and covariance. Orthogonality limits covariance. Standardization equates variance across features. Put together, each feature makes a roughly equal contribution to total variance, as do your components.
A: Well, clearly, if you were to perform PCA on a dataset, and then perform PCA on the result, you wouldn't get any benefit over just performing PCA once. So if your original dataset already had the properties of a PCA result (i.e. orthogonality), then applying PCA to it won't produce any further benefit. The more orthogonal a dataset is, the more it's already "PCA optimized", and the less further PCA helps. You should check the correlation matrix of the original dataset and see how much correlation there is between the variables. 
