4
$\begingroup$

I am new to machine learning, so forgive me if i am doing something absolutely absurd.

I have a classification task (~100 classes) and have about 2 million training data points in a 2000 dimensional space. Coordinates of data points are integers (discrete). All points have non-zero coordinates only for < 10 dimensions. That is, each point can be uniquely defined in < 10 dimensional sub-space.

If i use a Gaussian Mixture Model (GMM) for each class, i will end up with ~100 GMMs in a 2000 dimensional space. I feel that given the fact that each point is uniquely definable in less than 10 dimensional space, there can possibly be a better way of doing it.

What am i missing here?

Improve this question
$\endgroup$
3
  • $\begingroup$ Neural networks seem like a good fit. $\endgroup$
    – Marc Claesen
    Commented Mar 3, 2014 at 14:45
  • $\begingroup$ How about doing what you did, and using an l1 regularizer while doing estimation to enforce sparsity? $\endgroup$
    – elexhobby
    Commented Mar 3, 2014 at 14:48
  • $\begingroup$ @hrs how did you manage this? $\endgroup$
    – Mr. Unnormalized Posterior
    Commented Apr 4, 2018 at 19:16

1 Answer 1

Reset to default
1
$\begingroup$

Turns out GMMs are not good for modelling very high dimensional data because it increasingly becomes difficult to cluster points with more dimensions. Integer data was normalized. I reduced the dimensionalty using LDA followed by using training a GMM for each class. My feature space was incomplete and I ended training a CNN based classifier with even higher dimensional points and with even more data.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.