0
$\begingroup$

I want to predict roles based on technical skills column.I have column technical skills for which I have applied tf-idf to transform into numeric. I want to predict 5 roles for each observation.

Target Variable Role has 16000 unique values each observation has one role. (And I want to be able to predict the most likely 5 roles for each person)

Technical Skills are categorical (applied TF-IDF)

Can someone help me to find a solution. How can I approach this type of problem

Improve this question
$\endgroup$
5
  • $\begingroup$ How many data points do you have? $\endgroup$
    – Stephan Kolassa
    Commented Sep 28, 2021 at 15:14
  • $\begingroup$ @StephanKolassa around 100000 $\endgroup$
    – 10sha25
    Commented Sep 28, 2021 at 15:16
  • $\begingroup$ That's probably not enough data to learn a 16000-way classification. $\endgroup$
    – Arya McCarthy
    Commented Sep 28, 2021 at 15:17
  • $\begingroup$ That is very little data if you have 16,000 possible targets. You could do a classification using a Random Forest. However, I would assume that there are many technical skills levels for which there are more than five roles - all of these will be equally valid predictions. Consider combining some roles. $\endgroup$
    – Stephan Kolassa
    Commented Sep 28, 2021 at 15:18
  • $\begingroup$ How can I combine such roles because there are many combination of technical skills Can I use clustering here. Is it better than doing classification in this case. @StephanKolassa $\endgroup$
    – 10sha25
    Commented Sep 28, 2021 at 16:08

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.