0
$\begingroup$

I have a Dataframe that contains 2 columns:

'Skills' column - each cell contains a list of strings describing different technical and soft skills of a person, e.g: [Python,SQL,Java,Team Management,Teamwork,Communication skills]

'Status' column (target column to predict) - categorical column with 2 categories: Passed / Failed.

I would like to create a machine learning model to predict the 'Status' of a person using the list of skills as input. (Each person can have a different number of skills). I am new to the world of text classification, and I would like to know how to prepare the data for modeling. How should I handle the 'Skills' column and which libraries should I use?

Thanks.

$\endgroup$
0
$\begingroup$

You can start with scikit-learn, and try a naive bayes classifier:

import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import BernoulliNB

example dataframe

df = pd.DataFrame({"skills":[["Python","SQL","Java","Team Management","Teamwork","Communication skills"],["Java"]], "status": ["Passed","Failed"]})

some text normalization

translation = {"Team Management": "Team_Management","Communication skills":"Communication_skills"}

df["skills"] = df["skills"].apply(lambda row: [translation.get(x,x) for x in row])

df["skills"] = df["skills"].apply(lambda x: " ".join(x))

vectorizer

vectorizer = CountVectorizer(binary=True)

X = vectorizer.fit_transform(df["skills"])

y = df["status"]

model training

model = BernoulliNB()
model.fit(X, y)

inference

test= vectorizer.transform(["Python SQL Java Team_Management", "Java"])

model.predict(test)

that will return

array(['Passed', 'Failed'], dtype='<U6')
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.