Newly practicing data scientist here! I am currently stumped on a project and reaching out for some guidance:
I am working with the marketing team in our customer database. There is a small subset of customers who have exhibited a desired behavior (first purchased Product 1 and within 2 years purchased Product 2). The marketing team wants to target customers within our database that are similar to the desired customers before they made their purchase of Product 1. There are about 3,000 customers who exhibit the desired behavior (response = 1) and 300,000 have not purchased either product (response = 0). I have chosen 6 features, mostly based on purchase history. 5 are numeric and 1, Gender, is categorical.
I have attempted to use K Means clustering, by dropping the categorical feature and removing outliers. I did this in hopes that most of the response 1 customers would be clustered together. The model produced 3 clusters, and 80% of response 1 customers were indeed assigned to one cluster, along with 40% of response 0 customers (over 90K people altogether). However, I don't feel comfortable with the rigor of this attempt to create a "lookalike" audience or suggesting these nearly 90K response 0 customers to the marketing team as good leads.
I would like to try a classification algorithm like the random forest model but am coming up against several obstacles: a) the data is highly imbalanced and b) I only have one pool of customers so after training and testing the model, I can't figure out what data I would then use to classify and recommend for marketing. I considered under sampling to rectify the imbalanced data issue and then using the customers who were removed in under sampling to run the model on and classify. This seems very odd and counterintuitive to me.
Overall…I need help! Looking for the right direction. Clustering…classification..something else entirely? Any advice appreciated!