Predicting whether a potential sale will be won or lost

Question

I am currently working on a project using a sales system and trying to come up with a way to use the current pipeline of potential sales to predict the amount of product that will be sold in the future. I’m looking for advice on how to approach this problem and hopefully some resources to teach me what approach to use and why.

The sales system I’m using has historical data for opportunities (potential sales). Around 50,000 of the opportunities are “closed” meaning that they are either won or lost. I have around 1,000 “open” opportunities that have not yet been won or lost. Some variables that I have on each sale include the product (which is generally homogenous except for the amount), the amount, the salesman, the date, the time it was input into the system, the customer, and other data about the customer.

I understand that if I want to predict a dichotomous variable like win / lose then I should look at a logistic regression. However, I’m looking for general advice on how to

1. Predict the probability of each individual opportunity closing as won using the data I have (and how to tell if I've done it correctly).
2. Estimate the total amount of won opportunities for a period.

I found a similar question here Using a logistic model on the estimates of several other classification models but I’m hoping for a response that gives me a better idea of where to start. I’m comfortable using R or any other statistical software, but ideally I'd like some kind of book or other reference material that is as low-level as possible.

Answer 1

1. I deal with this sort of messy data a lot. If you have lots of time, there is no reason to not test on all the factors (Maybe not customers if you don't have repeat customers).

More than likely, you want more easily usable information. I suggest grouping your salesmen into "Best" "Medium" and "Acceptable" categories, or however many categories you want. This way your system will still work as you get new salesmen and as new salesmen come in, you can quickly grade them. Similarly, group the customers into "high quality we want lots of business from them" to "one-shot, small-time customers". This will help with regression, having simple categories.

To test if you've done it correctly is a different ballgame altogether, it really helps to think about the numbers that you get for a regression and think about the trendline that shows. If your "Not good" category of salesmen get better sales, it should automatically clue you in that something is wrong, since they are defined by not having the best sales records. If it shows no correlations at all, it doesn't mean you did it wrong, it might mean there is too much variance unaccounted for, so I can't really answer that part for you.

2. One of the problems I see in "Open" cases is messy paperwork can keep cases open for years. Make sure you clear stuff that is open past a reasonable time frame (you need to decide what makes sense for your situation).

I would set a time frame (monthly or quarterly) and compare the ratios of opens and closes; then looking at closes only, which ones are wins and losses. You can then start looking at baseline sales and trends and such. It is really difficult to accurately extrapolate sales data, and any recommendation I give starts with a long preamble about that.

---

In short, the regression is the easy part, cleaning your data takes all the work.

Answer 2

I have experience working on a similar data set. The positive response rate in my case is less than 0.01%. So it is pretty rare. I don't know what's the positive response rate in your case. I have tried hurdle models (because of the huge amount of negative responses in my sample), glm, glm mixed, and some regulated regressions, for example, elastic net and ridge regression, and also some machine learning methods, for example, regression trees, random forests, discriminant analysis. I've also tried to boost each single one of the above mentioned methods.

Specificly, for the logistic regression, to get predictions you use

    predict(fit, newdata,type='response')

This way you get the 1/0 predictions for your $newdata$.

I recommend you to try what I have tried in my case, and also methods mentioned by other people here. A good reference book is An introduction to statistical learning by Gareth James and others: you can easily find an online version on google.

Answer 3

If you plan to use a linear classifier (as logistic regression), you will get a score of belonging to each possible category (won/lost).

• In Python, you can use sklearn.linear_model.LogisticRegression.predict_proba from scikit-learn (Python). This is a nice example of how to visualize the probability of belonging to each class:
• In R, you can use predict() from glm.

Answer 4

One way to approach this problem is to look at steps to a sale. For example, the steps might be (1) appointment made to discuss (2) formal presentation (3) formal proposal written (4) counteroffer made, etc. as appropriate to this situation.

At each stage, you could determine the probability of a successful close. For example, at stage 2 the probability of a close might be 25%, at stage 4 it might be 75%. This gives you the basic model.

You can then look at modifiers -- e.g. type of product, type of client, which sales office, etc.

Answer 5

I recently worked on a very similar problem at work, and although I am unfamiliar with R, I hope I can contribute. Given your goals I would suggest you use a Heckman specification. These models proceed in two stages - firstly, they estimate the probability of a sale through use of a probit model, and then derive the expectation of the sale, conditional on a sale being made. I am suggesting this because the value of the sale is very obviously dependent on customer self-selection. Sorry, I don't know how to implement this in R, but these kinds of models are widely known so there should be a package.

Answer 6

Since you asked for lowest possible details, I think you want mathematics or concepts. I think all answers above are relatively high level in a sense that they discuss tools.

Fundamentally you wish to estimate a program $p$ that once you give it some input $\mathbf{x}$, it outputs a number in $[0,1]$ which is the probability of deal represented in $\mathbf{x}$ is going to be a winning one.

There are possibly infinitely many estimations of $p$ that work equally accurately on your learning set.

You need to use your domain knowledge to eliminate as much as possible from those infinitely many possible estimations of $p$ so that your space of possibilities shrink. Of course, you need to only remove bad ones (and keep good estimations).

Once you get a set of possible estimations of $p$ that you could find, you use them in a voting ensemble where their votes are weighted based on the probability of how likely they are to predict correctly.

As you see this is a very big problem. Simply choosing an algorithm that suggests to you estimations of $p$ limits your ability in seeing those estimations. There is no single algorithm to recommend in your case in my view. You need to explore your data, and try things out. Essentially, this is a game of guessing.

Personally I'd give Random Forests-based ensembles in Regression mode a try first. If tinkering with their parameters doesn't help, I'll then try various designs of Artificial Neural Networks as some of them are known to be Turing complete.

I think there is a reason why Deep Learning is heavily focused on Artificial Neural Network designs. I think it can do more than any other family of learnins methods.

Answer 7

Look, It is very simple Problem. You have not Large but good Dataset. So, the approach will be like that.

Clean Your dataset( like Outliers and Missing Value ) -> Split Your DataSet into Training and Test Set -> Build couple of Models like Logistic Regression, SVM, Random Forest or ANN -> Evaluate their Performance on Your Test Set and Calculate Accuracy and Area Under ROC Curve for each Model -> Model having highest accuracy and Highest AUC will be your final model which you can use for predicting Win/Loss for the unknown example.

This is the right approach for every classification problem. Now, I am going to publish my paper soon :-) . So that will be the best reference for you.