Situation: We are utilizing an application that transcribes phone calls into text and identifies when certain phrases (that we define) are said. We then enter logic for "categories" that use the metadata from the call, plus the presence of a phrase, to categorize the call. A single call will have many categories.
Problem: We want every call to have its primary purpose identified using a category. I can query the db and pull all of the categories for every call and find the ones that do not have one of the primary purpose categories. We need to define more content (phrases) for those calls. I want a better way to identify samples of calls to listen to, rather than just doing it at random. I would like to separate the calls into clusters and then sample from each cluster, where a cluster would be calls that have a lot of similarity in the entire list of categories applied to the call.
I want to implement this in R.
The data I'm dealing with looks like this:
call_id | Category
1 | A
1 | B
1 | C
2 | A
2 | D
Desired output: Thinking about other tests and visuals there are several types of output that would work.
- Something like a comparison of means with each call assigned to one or more groups.
- A correlation matrix, and with some trial and error I could create groups based on a certain correlation coefficient threshold.
- Some kind of clustering algorithm that would magically group them based on some kind of similarity score.
- Something like a classification tree, but I don't have labels in advance.
- I'm open to other ideas as long as I get the desired output.
The point of this is to make the time listening to calls more efficient, not to maximize the accuracy of the clustering algorithm. I want something that can be implemented quickly and not require a lot of data prep or cleanup afterwards.