Based on your comment, I believe there could be two, not necessarily related, things at play:
- Your principal components do not explain enough of the variance; and
- Your groups cannot be distinguished from each other given PCs.
The first issue may arise when your data is "spherical", i.e., when the off-diagonal elements of your variance-covariance matrix (covariances) are zero or very small compared to the diagonal elements (variances).
In this case, rotating the data space (as PCA does) will not help you increase variance in a new direction.
Furthermore, if the space is high-dimensional, in the case of spherical distribution, the first few PCs capture a very small proportion of the total variance.
As a demonstration, I simulated two multivariate distributions, one with a random variance-covariance matrix (with non-zero covariances; x.1), and one with a diagonal variance-covariance matrix (with zero covariances; x.2)
set.seed(2022-08-03)
N <- 1000
d <- 5
## MvN with a random variance-covariance matrix
A <- matrix(runif(d ^ 2) * 2 - 1,
ncol = d)
Sigma <- t(A) %*% A
# diag(Sigma) <- diag(Sigma)*0.95
x.1 <- mnormt::rmnorm(N,
rep(0, d),
Sigma)
## MvN with a diagonal variance-covariance matrix
x.2 <- mnormt::rmnorm(N,
rep(0, d),
diag(runif(d)))
Here are the distributions of the x.1 and its 5 principal components:
As you can see, the PCA algorithm rotates the data such that the PCs are uncorrelated.
Looking at the (commulative) variances explained by PCs is illuminating:
RC1 RC2 RC3 RC4 RC5
SS loadings 2.49 1.25 1.11 0.09 0.07
Proportion Var 0.50 0.25 0.22 0.02 0.01
Cumulative Var 0.50 0.75 0.97 0.99 1.00
Proportion Explained 0.50 0.25 0.22 0.02 0.01
Cumulative Proportion 0.50 0.75 0.97 0.99 1.00
Which shows that the first component(s) have higher contributions to explain the variance, and the first 3 PCs capture decent amounts of the total variance: respectively, 50, 25, and 22%; in total 97%.
On the other hand, if the raw data has no covariances (x.2), the PCA rotation does not make much of a difference:
For x.2, the principal components capture the same amount of variance (here: $\frac{1}{d} = 0.2$):
RC1 RC2 RC4 RC3 RC5
SS loadings 1.0 1.0 1.0 1.0 1.0
Proportion Var 0.2 0.2 0.2 0.2 0.2
Cumulative Var 0.2 0.4 0.6 0.8 1.0
Proportion Explained 0.2 0.2 0.2 0.2 0.2
Cumulative Proportion 0.2 0.4 0.6 0.8 1.0
Thus, the first 3 PC3 only explain 60% of the variance in the data.
As the dimensionality of the data increases, the proportion of variance explained by the first components of x.2 decreases faster than x.1.
You can verify this if you increase dimensionality of the simulated data (say, d <- 10), and look at the PCA outcomes (specifically, Proportion Var and Cumulative Var of the first three PCs) for the new x.1:
RC1 RC2 RC3 RC4 RC5
SS loadings 2.88 2.43 2.41 2.30 2.09
Proportion Var 0.14 0.12 0.12 0.11 0.10
Cumulative Var 0.14 0.27 0.39 0.50 0.61
Proportion Explained 0.24 0.20 0.20 0.19 0.17
Cumulative Proportion 0.24 0.44 0.64 0.83 1.00
And the new x.2:
RC1 RC5 RC4 RC3 RC2
SS loadings 1.20 1.19 1.18 1.16 1.15
Proportion Var 0.06 0.06 0.06 0.06 0.06
Cumulative Var 0.06 0.12 0.18 0.24 0.29
Proportion Explained 0.20 0.20 0.20 0.20 0.20
Cumulative Proportion 0.20 0.41 0.61 0.80 1.00
The second issue is because your groups are not linear separable in the base space (raw data) or the PCA reduced space.
A very simple example of this is when the means are close to each other.
To figure this out, I would suggest making and eyeballing pairplots of yor raw and reduced data, colored per group.
I believe this tutorial will help you in making such plots.
