I'm working on a binary classification problem using 10 predictors. I have 4508 data points. I am required to have a prediction scheme which achieves an accuracy of 80% or better. However I am discouraged by the appearance of my data. It seems like the classification problem is not possible with what I have been given. Here are two group/scatter plots over two pairs of predictors. (It's same over all pairs.)

Predictors 2 and 10Predictors 2 and 3

What do you recommend?

Update: Here is a non-linear embedding of a subset of the data into 2 dimensions. Red colours are class one and blue colours are class zero. (Sorry for the picture size)

Diffusion Map

And another one using another method: t-SNE

  • 1
    $\begingroup$ It might be well separated in the original feature space $R^{10}$. If you do want to visualize the data in low-dimensional space, it is suggestive to use the first few principal components. $\endgroup$
    – Zhanxiong
    Oct 31 '16 at 4:13
  • $\begingroup$ One of the great things about machine learning algorithms is that they can find patterns that we humans have trouble seeing. @Zhanxiong's idea is a good way to look at the data, but you can also try some algorithms and see what you get. I guess the answer to your question is not to get discouraged until you've tried a lot of things. $\endgroup$ Oct 31 '16 at 4:35

Not everything can be predicted, if this were the case I'd stick a random forest on the stock market and retire :)

Might I suggest you try feature engineering? For example, if you have names like Mrs Jones, Mr Smith, Miss Johnson... try using delimiters to create a new variable for the title of their names, then rerun the models.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.