We currently applied PCA to a set of variables, and noticed that our dataset actually contains two "motifs". To explain, let say we have the variables
A, B , C, D, E, ...... Z. When inspecting the PCA rotations and the correlation between variables we noticed that the dataset actually consist of two main groups, like so (We use binary indicators, either they have the variable property or they don't):
group 1: important variables
A = 1, B = 1, C = 0, D = 0
group 2: important variables
A = 0, B = 0, C = 1, D = 1
However when we now fit new data to this PCA model this will obviously score a mix of both groups high, for example
A = 1 B = 1, C = 1, D = 1. Therefore we are wondering whether a similar method exists where the variables keep their dependence (but still a method that seeks for explaining most variation).
Is there a method just like PCA that incorporates dependence between variables?