I would like to use machine learning to determine the race of an individual based on things like the first letter of his/her name, my problem is that the data I have is distributed in the following way:

COUNT RACE            
92742 African             
12    Asian
43349 Mixed race               
327   Foreign National       
4588  Indian              
11179 White           

So the data is very skewed in terms of volume towards African and Mixed race. When training my learner will this affect the learning and what should I do to combat this? Also instead of the first letter of the name (A-Z) should I use the number representation of the alphabet letter (1-26) or even a representative value between 0 and 1?

  • $\begingroup$ you should try to collect more data for the underrepresented classes. right now the imbalance is too severe (especially for the Asian class). For mild imbalance you could use algorithms such as SMOTE $\endgroup$ – Antoine Mar 17 '17 at 9:13

If you really plan to classify based on "names" alone, I would suggest character level language modelling of names for each classes, however classifying based on first letter alone would be such a vague idea.

  • $\begingroup$ I'm planning to classify based on loads of features. Your answer has nothing to do with my question. $\endgroup$ – Superdooperhero Mar 17 '17 at 9:10
  • $\begingroup$ @Superdooperhero you're wrong. Character level recurrent neural networks such as LSTMs could give good results in your case $\endgroup$ – Antoine Mar 17 '17 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.