If the features are really independent with each other, then a clustering algorithm will just segment our sample space at random. Not very informative as there is not underlying structure to be discovered.

That said, if we are dealing with binary variables (0/1s) then we could use something like logistic PCA where one minimises the Bernoulli deviance; the relevant reference for us was Dimensionality reduction for binary data through the projection of natural parameters by Landgraf & Lee (2020). Following that, we get the PC score and then cluster the scores. Logistic PCA is implemented in the R package logisticPCA if you want to try it yourself. Prior to that, Tipping's (1998) Probabilistic Visualisation of High-Dimensional Binary Data was an early attempt much closer to the now-classical view of a latent Gaussian model for PCA.

If the features are really independent with each other with no underlying structure, then a clustering algorithm will just segment our sample space arbitrarily. Unsurprisingly, that won't be very informative, as there will be no underlying structure to be discovered. The clustering algorithm might even appear "stable" for a given choice of hyperparameters, despite the absence of a meaningful grouping.

That said, if we are dealing with binary variables (0/1s) then we could use something like logistic PCA where one minimises the Bernoulli deviance; the relevant reference for us was Dimensionality reduction for binary data through the projection of natural parameters by Landgraf & Lee (2020). Following that, we get the PC score and then cluster the scores. Logistic PCA is implemented in the R package logisticPCA if you want to try it yourself. Prior to that, Tipping's (1998) Probabilistic Visualisation of High-Dimensional Binary Data was an early attempt much closer to the now-classical view of a latent Gaussian model for PCA.

Finally, do note that aside the Bernoulli deviance advocate above, we can define multiple other meaningful similarity measurements between binary (or mixed-typed) entries. Some of the obvious one are the Cosine similarity and Jaccard similarity (or Gower's distance for mixed type data). All of which allow us to create a distance/dissimilarity matrix $D$ directly and use a clustering algorithm directly on $D$.

(Thank you to Nick Cox and Christian Henning for calling out shortcomings to my initial answer.)

If the features are really independent with each other, then a clustering algorithm will just segment our sample space at random. Not very informative as there is not underlaying structure to be discoved.

That said, if we are dealing with binary variables (0/1s) then we could use something like logistic PCA where one minimises the Bernoulli deviance; the relevant reference for us was Dimensionality reduction for binary data through the projection of natural parameters by Landgraf & Lee (2020). Following that, we get the PC score and then cluster the scores. Logistic PCA is implemented in the R package logisticPCA if you want to try it yourself. Prior to that, Tipping's (1998) Probabilistic Visualisation of High-Dimensional Binary Data was an early attempt much closer to the now-classical view of a latent Gaussian model for PCA.

If the features are really independent with each other, then a clustering algorithm will just segment our sample space at random. Not very informative as there is not underlying structure to be discovered.

That said, if we are dealing with binary variables (0/1s) then we could use something like logistic PCA where one minimises the Bernoulli deviance; the relevant reference for us was Dimensionality reduction for binary data through the projection of natural parameters by Landgraf & Lee (2020). Following that, we get the PC score and then cluster the scores. Logistic PCA is implemented in the R package logisticPCA if you want to try it yourself. Prior to that, Tipping's (1998) Probabilistic Visualisation of High-Dimensional Binary Data was an early attempt much closer to the now-classical view of a latent Gaussian model for PCA.