I am new here - and relatively new to statistics, data mining and R. I am trying to understand why my data is not clustering correctly - or if I am reading it wrong. Shortly about the project:
My data points are behavior of online gaming users over time (one observation for each user). I am trying to cluster a matrix containing 5000 observations. So far I have 4 dimensions that correspond to average games/day, interval between days, score, friends - all of these happened in the first 2 months of user's online presence. So it looks approximately like:
avg_g_day avg_interval score friends 1 8.5 6 6050 0 2 48.1304 1.8636 90530 0 3 70.0702 1.0714 293520 2 4 25 1 5710 4 5 3.75 10 10900 0
I am trying cluster this using all possible methods in R. So far I have not had success. I first scale my data using scale() and then fit the clusters. When I try kmeans (which would mean Euclidian distance, in my understanding), I get intersecting clusters upon visualization with clusplot. When I am trying to find the best k, I get that the best k is 2, which seems quite un-commonsense with 5000 points and 4 dimensions. I have tried other methods and other distance metrics, so far to no avail - I have used Mclust() and hclust. In each dimension the distribution is not uniform and I do not get warnings that best number of clusters is 1. Then why does the kmeans or pam method give me clusters that intersect on the 2 first principal components graph (clusplot)? Is there a way to separate them?
Since I am not sure which details could provide more insight into my problem, I will post them if questions arise. Thank you for any help.