I’m feeling a bit unsure about what I’m working on as I have never dealt with this sort of data before, so I could use some feedback.
I have a large dataset that I have clustered. My approach was (1) scaling (between 0 & 1), (2) PCA, and then (3) K-means clustering. I also applied HDBSCAN as an experiment. The results seem reasonable, but I would like to understand more about the data, whether what I have done is acceptable, and whether there are other things that I should consider.
The variables look like the following, although this is fake data and I really have ~25 variables:
I am used to dealing with gaussian and occasionally slightly skewed data. These data are heavily zero inflated, extremely skewed, and some (e.g. #3) are Bernoulli distributed. My questions:
- What do I need to keep in mind when dealing with this sort of data?
- What are the implications for the data types when applying feature scaling? PCA?
- What are the implications for choosing a distance metric when clustering?
- Any other caveats?