I need some help finding a better way to statistically group data for a project I am working on.
I have a data set of individuals with different skill sets. Each individual can have only 1 or several different skills. Some of the individuals can even have up to 40 skills that they work with. I am trying to find a more statistical way to find groups of agents based on the skill while also keeping a large percent of the data. Here is what the data looks like:
Individual ID Skills 1 1,2,3,4 2 2,3,4 3 2,3 4 3,4
So ideally, it would be nice to find the maximum number of skills that group people together and maintain the most number of records. What would not be ideal is taking all of the skills because some of the skills may have very low pct of the individuals or records. Is there a more statistical approach to grouping these people together into one large bucket (for example, we use 2,3,4 because its 90% of the agents and 95% total records)
Another good question that I have still yet to figure out is what percentage of coverage is good.