I am trying to find an individual agent's score based on customer scores (raters) while the customer only rates their overall experience.
The customer enters a shop and meets a service agent number one. Then he moves onto service agent two and so on. Generally, a customer meets 4 to 6 agents one after another and once he is done with all the transaction he is asked his overall experience of all the agent interactions on a scale of 1 to 5. It is possible to know which all service agents the customer met and tie the overall score to all those service agents
It is optional for customers to leave feedback. Moreover, there are multiple service agents, with different shift routines, close to 200, that are working in different departments which implies that each customer is unique but service agents are randomly assigned.
Eg - Customer no. 1 meets service agent A,B,C,D and E, while customer no. 2 meets B,D,E,G and H.
- I am trying to ascertain the rating of each service agent based on the overall customer store
- Additionally, it would be great if I could somehow compare service agents across departments
I have been assuming that each service agent is equally liable for the overall experience score and thus I have been doing the following
- Assign the overall experience score to each service agent who met that customer
- Find the average of all such customer scores for that service agent
Service Agent Score Table
Service Agent ║ Average Score
║ A ║ 3.67
║ B ║ 4
║ C ║ 3
║ D ║ 5
║ E ║ 2.5
I have also searched stack exchange and come across terms such as Cronbach's alpha, Cohen's Kappa and Fleiss Kappa but I do not think they apply to my case. I think this way of rating plus ANOVA can do the trick as can be found here which is somewhat similar to my problem.