2
$\begingroup$

I am working with a typical ML problem, trying to separate 2 classes in a supervised manner. I wanted to ask at which point you decide the data is not descriptive enough to solve the problem, and if there is anything furhter you can do.

My problem is that ive been given throught my education may tools to solve problems, and improve on existing algorithms, but not told what to do when the data doesnt provide enough variance.

The data set is described by around 20 continuous feautres with binary labels for each case. I have used T-SNE (seen below) to visualise a dimensionality reduced version of my data set.

TSNE data visualisation

From this, and PCA plots I have made, I would say the data is certainly not linearly separable, and not non-linearly separable either.

What steps can I take to help separate the classes? Will combining features aid this process (ratios between two etc) or is my understanding correct that these relationships will be found if important.

Any advise would be appreciated.

Thanks!

EDIT: PCA PLOT

enter image description here

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
4
$\begingroup$

PCA plots I have made, I would say the data is certainly not linearly separable, and not non-linearly separable either.

Note that, using PCA to map to 2D space will lose information in the data. It is possible the data "can be" separated well, but PCA 2D does not look good.

Even with the plot shown, there is still "hope", two different types of the data are "clustered", think about K nearest neighbor algorithm. It should give some "reasonable" performance.

Also, it is wrong to say they are not non-linear separable. If you do not have two overlapping points with different color, I can make a non-linear classifier with 100% accuracy.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your response. The PCA plots I have made are potted on top of each other (I have added one above). The first plot is a TSNE, and I agree a KNN would possible work here based on "clusters" seen. Im not sure if its possible to map TSNE projections on a test set however in order to reduce to the same space. Any advice on this would be appreciated :) $\endgroup$
    – JP1
    Commented Jan 29, 2018 at 8:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.