I am new to machine learning, so forgive me if i am doing something absolutely absurd.

I have a classification task (~100 classes) and have about 2 million training data points in a 2000 dimensional space. Coordinates of data points are integers (discrete). All points have non-zero coordinates only for < 10 dimensions. That is, each point can be uniquely defined in < 10 dimensional sub-space.

If i use a Gaussian Mixture Model (GMM) for each class, i will end up with ~100 GMMs in a 2000 dimensional space. I feel that given the fact that each point is uniquely definable in less than 10 dimensional space, there can possibly be a better way of doing it.

What am i missing here?