I have $N$, $1024$-dimensional vectors. I want to cluster them by some similarity. Given the high dimensionality, standard metrics won't work. I tried a few Approximate Nearest Neighbor implementations which use randomized kd-trees or priority search k-means to find nearest neighbors. I don't trust their results either, because they depend a lot on parameter configuration.
Now, I know the number of clusters in this dataset because I know the number of classes , which is 55. As I was thinking, I had this idea, that I could just take a random weight matrix $W$ of size $1024$ x $55$, that will take as input my vectors and output a $55$ dimensional vector, and I'll apply $argmax$ on this $55$ dimensional vector and just use that value as the cluster $id$ . The $W$ , once randomly selected, remains the same for every input vector. Because I only care about clustering similar vectors and not about their exact class, I think this should work well. I guess this is very similar to cosine similarity, but instead of distance, I get a cluster id, because I use this random $W$ as a "judge" of similarity...Similarity w.r.t $W$, in other words.
Now, I'm an engineer and while this looks at first glance to be okay, I know things can't be that easy..I mean people have been researching high-dimensionality clustering and search for a very long time, and it is a hard problem. So I know I either made some stupid logical flaw above, or that there is some good theoretical reason not to do it this way. I want to know what I'm thinking wrong here, and why.