When can I say there is no clusters? I am trying to do cluster analysis for a very small data set (<100) in higher dimensional feature space. I tried K-means and Hierarchical clustering, but I found no 'elbow' and also the silhouette score was never above 15%. To be honest, I do not know if I could use other algorithms as I am not sure which is best for my use case.
I tried to use PCA, but I can come up with examples where the 2D embedding has clusters but not the original feature space, and vice versa.
My question is:
When can I safely conclude that there is no clusters in my data?
Update
In this notebook, I try to generate examples where PCA is misleading.
 A: Generally, as mentioned in the comments, there are different concepts of what a cluster is, and whether "there are clusters" or not depends on your cluster concept. I wrote something about this as Chapter "Clustering strategy and method selection" in the "Handbook of Cluster Analysis", see here:
https://arxiv.org/abs/1503.02059
Here's one possibility to decide that there is no evidence for clustering. This requires you to specify two things, which should be related to the cluster concept of interest:

*

*A null model that models your idea of homogeneous data (for example a Gaussian distribution, or a uniform distribution, as used for the gap statistic). The null model should be chosen so that non-clustering features of your data are matched, for example by using the estimated mean and covariance matrix; see also https://link.springer.com/article/10.1007/s11222-015-9566-5


*A statistic that measures the degree of clustering. For $k$-means this can well be the $k$-means objective function, but it might also be something else such as the Average Silhouette Width.
You can then generate data sets from the null model, cluster the data into two (or more) clusters, compute the statistic lots of times (2000, say), which simulates a distribution of the test statistic under the null model, and compare the value(s) that you get on your data to the null distribution. If your data don't give you significantly better values, there is no evidence for clustering.
By the way, this is the idea of the gap statistic (using a uniform null and the log objective function of $k$-means), which gives you a formal rule for choosing "no clustering" (i.e., number of clusters 1) as opposed to the Silhouette Width or the elbow rule.
