How to choose data for training a predictive model for attrition prediction

Question

Trying to build a predictive model for attrition prediction at service desk/call center.

Have daily data on the following parameters:

1.Call quality - QTM (0-100%), 2.No. of calls - Calls(Number) 3.Attendance 4.Customer feedback(1/0) Q1,Q2 (0-100%) for both, agents who left the job and for the ones who are still there, for a duration of 6 months.

Aim: to predict agents tendency/probability of staying/leaving based on his/her daily performance.

Doubts i have, 1. how should i use the data to train the model(logistic regression)

should it be trained based on the avg of the parameters taken over a duration of 6 months.

**if so can we test the daily metrics based on a model which is trained using mean of the parameters for 6 months.

Please advise.

this is my first attempt at making a predictive model,i have gone thru various case studies/models such as the titanic survival model using logistic regression,Wisconsin DEWS model.

I decided to model using the weekly aggregates of the the two populations(attrites and Non-attrites).

The Data Set (approx 5 months data,with weekly aggregates of the two populations i.e Attrites and Non-Attrites.) AW1 : Week1 Aggregates of the performance metrics for Attrites NAW1: Week1 Aggregates of the performance metrics for Non-Attrites

Post this i ran a logistic Regression on 80% of this data-set and kept aside the other 20% for testing. Results of the logistic regression:

and then i used the predict function on the 20% of the data which contained 3 data points for both attrites and Non-attrites,so to be 100% accurate the model should have predicted 3 as attrites and 3 as Non-attrites but the correct prediction is 5/6 that is one wrong prediction out of 6.

Please help me in interpreting the meaning of the results of the model all the z values are zero im not sure what that signifies.

Googled a little regarding the z values = 0 issue and came across some posts on stackoveflow that suggested using "bayesglm" instead of "glm" did that and the results are good at the first look but being a newbie in the field i would like you to guide me with respect to the statistical significance of the issue and is the model really as good as the results of the "bayesglm" or is it just by fluke.

the model gives a 100% accurate prediction on the test set now 6/6.

Answer 1

I would aggregate the data to weekly aggregate numbers, assuming that great / bad agents have some what consistent call center performance over the six months. Sometimes aggregating erases the effects of outliers before they can be classified as such. This would account for shifts in performance across the total 6 month period as well.

When it comes to sampling using 80% of data points to develop model and 20% to validate would be a good start. Can adjust those numbers depending on how big a data set you are dealing with.

I utilize Iowa State papers some times. Here is a good one on the basics (pdf).

Hope You have fun!!

---

Update: Just so we are clear you are aggregating by week per customer service rep right?

Both models don't fit good. You can tell variable fits using the coefficients section of the results. Significant variables have the stars next to there P value (more stars equals more significant typically and lower P value). Based on that none of your variables are actually

It's good that you are comparing the model vs actual results. ROC curves capture the model differences pretty well. Try running this and post what you get.

library(pROC)
g <- roc(admit ~ prob, data = mydata) 
plot(g)

Update: Its weekly aggregates, population wise(i.e the attrite population and the active population),didn do it agentwise because we will have cases when a agent leaves when he was at his peak performance but those are exceptional cases so i thought it would be better to compare the two populations, please advise if that's not the correct way of thinking

SO AW1 is first weeks performance metric aggregates for Attrites, similarly NAW1 is first weeks performance metric aggregates for Non-attrites/Active agents.

Ran the "step" fuction(Selects a formula-based model by AIC) on the bayesglm model and the results are as below;

---

Aggregating all the agent results together will mean you essentially are over fitting to match the total population metrics and not the agent's performance. Recommend that you tie in the agent level results. You mentioned there being a chance that a great agent leaves unexpectedly but for a well run unit that should be a rarity. Also, recommend you change model family parameter to

family = binomial(link = "probit")

This should give you probability of default for each agent. This would

Answer 2

In engineering terms, the "fit an equation" approach alone often becomes voodoo. It does not answer "why". It does not inform effective actions.

So if I manually entered the data:

Subject Calls   QTM Attend  Q1_rev  Q2_Rev  Status
AW1 24.5    98.5    71.21   94.44   90.35   1
AW2 28  96.06   70.96   97.22   93.08   1
AW3 30  95.23   64.51   97.7    94.02   1
AW4 27  96.4    65.57   97.14   93.9    1
AW5 29  97.14   76.47   97.22   94.23   1
AW6 27  90.48   81.79   97.22   94.33   1
AW7 29  97.89   59.34   97.1    94.38   1
AW8 28  92  97.78   97.28   94.5    1
AW9 29  97.17   91.11   98.08   94.72   1
AW10    27  86.49   97.72   98.14   94.9    1
AW11    26  89.8    95.74   98.17   94.79   1
AW12    27  93.95   97.78   98.44   95.08   1
AW13    27  82.14   92.85   98.65   95.49   1
NAW1    26  95.49   87.8    99.27   94.74   0
NAW2    30  94.19   84.83   98.3    94.65   0
NAW3    32  94.17   80.44   98.66   94.67   0
NAW4    29  96.41   81.05   98.71   95.06   0
NAW5    30  96.49   85.46   98.6    95.03   0
NAW6    28  94.84   88.69   98.69   95.24   0
NAW7    29  94  91.22   98.82   95.81   0
NAW8    29  95.47   88.75   98.57   95.84   0
NAW9    29  96.84   84.88   98.47   95.82   0
NAW10   29  93.8    93.56   98.38   95.96   0
NAW11   27  94.02   87.07   98.35   95.96   0
NAW12   28  97.82   84.65   98.33   96.01   0
NAW13   29  96  86.62   98.36   96.09   0

And then use the following code to look at the variables in terms of "Staus":

#housekeeping
rm(list = ls())

#load libraries
library("vioplot")


#set working directory
setwd("C:/Users/mrmunroe/Desktop/")

data <- read.table(file="Book1.csv",header=TRUE,sep=",")

ind1a <- as.numeric(subset(data$Calls,data$Status==1))
ind2a <- as.numeric(subset(data$Calls,data$Status==0))

ind1b <- as.numeric(subset(data$QTM,data$Status==1))
ind2b <- as.numeric(subset(data$QTM,data$Status==0))

ind1c <- as.numeric(subset(data$Attend,data$Status==1))
ind2c <- as.numeric(subset(data$Attend,data$Status==0))

ind1d <- as.numeric(subset(data$Q1_rev,data$Status==1))
ind2d <- as.numeric(subset(data$Q1_rev,data$Status==0))

ind1e <- as.numeric(subset(data$Q2_Rev,data$Status==1))
ind2e <- as.numeric(subset(data$Q2_Rev,data$Status==0))



par(mfrow=c(1,5))
vioplot(ind1a,ind2a,
        names=c("Calls | Status1","Calls | Status0"),
        col="Red")

vioplot(ind1b,ind2b,        
        names=c("QTM | Status1","QTM | Status0"),
        col="Orange")

vioplot(ind1c,ind2c,        
        names=c("Att | Status1","Att | Status0"),
        col="Green")

vioplot(ind1d,ind2d,        
        names=c("Q1r | Status1","Q1r | Status0"),
        col="Blue")

vioplot(ind1e,ind2e,        
        names=c("Q2r | Status1","Q2r | Status0"),
        col="Violet")

Then I get the following image. When I, as a human look at it, and I consider that if "Status 0" is someone who attritioned, then it looks like those who leave are the better and brighter. If that is the case then it is diagnostic. My sincere hope is that this is not the case, but unlike voodoo it makes a difference between data and understanding.

The fact that we have a predictive model doesn't mean that we understand why. Both descriptive models (answer what, like the table) and predictive models (answer how, like the equation)are things computers can do. Diagnostic tools, tools that actually empower change and improvement have to have "why" as the answer. Currently the only such tool is a human mind.

There is a program called JMP that has an amazingly powerful tool called a "Variability plot". It is straightforward to use and, at Intel, is the primary graph used in the company. It is a strong and effective tool for reducing very complex seeming problems to their basic roots. On of the things that they do is handle multi-level nesting. I need to learn more about lattice graphics and plotting in R to be able to make a 3-level variability plot in R.

Answer 3

I understand that you've already checked an answer, but I suggest an alternative approach all the same. Why not use random survival forests to predict the conditional cumulative hazard of attrition? This method has the benefit of having few assumptions about the underlying hazard function, and does not rely on the proportional hazards assumption. In addition, The method is implemented in randomForestSRC (formerly within randomSurvivalForests). The limitation of this method is that you would need both tenure of the workers as well as an indicator for whether there was attrition even for each worker. I would prefer this method over logistic regression because time is not really discrete in this process; it is at least approximately continuous. Another benefit of random survival forests is that it automatically considers complex interactions among your model features.