Calculation of the GINI coefficient,Accuracy and AUROC for credit scoring using Python code

Question

I have the following data and I want to compute the GINI and Accuracy for model validation purposes. But I tried to calculate the GINI and Accuracy using Python code, but it seems incorrect. I would like to compute the AUC, GINI and Accuracy by calculating the cumulative no of borrowers, cumulative no of goods, and cumulative no of bads.

Because I want to implement this in Microsoft excel and Python, hence trying to calculate but no success

Below are the codes:

# code 1
import matplotlib.pyplot as plt
from sklearn.metrics import roc_curve, auc

data = {
  "Decile": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  "No. Borrowers": [100, 300, 200, 300, 600, 200, 700, 800, 900, 1000],
  "Good Borrowers": [80, 160, 140, 220, 500, 1000, 560, 640, 1500, 800],
  "Bad Borrowers": [20, 140, 60, 80, 1000 ,1000 ,1400 ,1600 ,7500 ,200]
}

good_borrowers = data['Good Borrowers']
bad_borrowers = data['Bad Borrowers']

total_borrowers = [good_borrowers[i] + bad_borrowers[i] for i in range(len(good_borrowers))]
cumulative_good_borrowers = [sum(good_borrowers[:i+1]) for i in range(len(good_borrowers))]
cumulative_bad_borrowers = [sum(bad_borrowers[:i+1]) for i in range(len(bad_borrowers))]

cumulative_good_borrower_ratio = [cumulative_good_borrowers[i]/total_borrowers[i] for i in range(len(total_borrowers))]
cumulative_bad_borrower_ratio = [cumulative_bad_borrowers[i]/total_borrowers[i] for i in range(len(total_borrowers))]

fpr,tpr,_ = roc_curve(data['Decile'], cumulative_good_borrower_ratio)
roc_auc = auc(fpr,tpr)
gini = (2 * roc_auc) -1

print("AUC: ", roc_auc)
print("GINI: ", gini)


#code 2: 
import numpy as np
import matplotlib.pyplot as plt

# Create a data frame to store the number of borrowers, good borrowers, and bad borrowers in each decile.
data = {
  "Decile": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  "No. Borrowers": [100, 300, 200, 300, 600, 200, 700, 800, 900, 1000],
  "Good Borrowers": [80, 160, 140, 220, 500, 100, 560, 640, 150, 800],
  "Bad Borrowers": [20, 140, 60, 80, 100, 100, 140, 160, 750, 200]
}

# Calculate the ROC curve.
fpr = []
tpr = []
for decile in range(0, len(data["Decile"])):
    good_borrowers_in_decile = data["Good Borrowers"][decile]
    bad_borrowers_in_decile = data["Bad Borrowers"][decile]
    total_borrowers_in_decile = good_borrowers_in_decile + bad_borrowers_in_decile
    fpr.append(bad_borrowers_in_decile / total_borrowers_in_decile)
    tpr.append(good_borrowers_in_decile / total_borrowers_in_decile)

# Plot the ROC curve.
plt.plot(fpr, tpr)
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("ROC Curve")
plt.show()

# Calculate the AUC.
auc = np.trapz(tpr,fpr)
print("AUC:", auc)

# Calculate the accuracy.
accuracy = (sum(data["Good Borrowers"]) + sum(data["Bad Borrowers"])) / sum(data["No. Borrowers"])
print("Accuracy:", accuracy)

# Calculate the Gini coefficient.
#gini = 1 - np.sum((np.array(data["No. Borrowers"]) * (np.array(data["No. Borrowers"]) -1))) / (np.prod(np.array(data["No. Borrowers"])) **2)
def gini(data):
    borrowers = np.array(data["No. Borrowers"])
    if 0 in borrowers:
        return None
    gini = 1 - np.sum((borrowers * (borrowers -1))) / (np.prod(borrowers) **2)
    return gini
gini(data)
print("Gini coefficient:", gini)

Data:

Decile	No. Borrowers	Good Borrowers	Bad Borrowers
1	100	80	20
2	300	160	140
3	200	140	60
4	300	220	80
5	600	500	100
6	200	100	100
7	700	560	140
8	800	640	160
9	900	150	750
10	1000	800	200

I hope that helps!

Answer 1

Given that you only have summary data (not the full dataset) then the scikit-learn metrics package will not work as expected. I found this useful blog that I think may help you understand how that function works. The blog uses an example very similar to your question.

Also, I need 50 creds to write comments on posts, so I wasn't able to ask for clarification. However, based on your data, it seems that you are trying to calculate the fpr and tpr of a good/bad borrower classifier. My interpretation is that the "No. Borrowers" column is the number of borrowers classified as good borrowers by the model given that you use a threshold of selecting the top $i^{th}$ decile, while the other two columns are the true number of good and bad borrowers at that decile.

If that is the case, then we must make a few corrections to the code. Remember that $FPR = \frac{FP}{FP + TN}$ therefore at a given threshold we must divide the total number of accepted bad borrowers (the cumulative sum until the $i^{th}$ threshold) and divide by the total number of bad borrowers.

similarly, the tpr may be computed using the formula: $TPR = \frac{TP}{TP + FN}$

Your accuracy can be calculated with the following formula $Acc = \frac{TP + TN}{Total\_population}$. In other words, your accepted positives (cumulative sum of good borrowers until $i^{th}$ threshold), plus your denied negatives (sum of bad borrowers beyond $i^{th}$ threshold), divided by you total number of borrowers.

Here is my implementation of your code. I will ignore code #1 since I believe code #2 is closer to a good answer. I have also taken the liberty to make the code a bit cleaner by using more built-in tools from numpy and pandas.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Create a data frame to store the number of borrowers, good borrowers, and bad borrowers in each decile.
data = {
  "Decile": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  "No. Borrowers": [100, 300, 200, 300, 600, 200, 700, 800, 900, 1000],
  "Good Borrowers": [80, 160, 140, 220, 500, 100, 560, 640, 150, 800],
  "Bad Borrowers": [20, 140, 60, 80, 100, 100, 140, 160, 750, 200]
}

df = pd.DataFrame(data)

total_population = df['No. Borrowers'].sum()
df['TP'] = df['Good Borrowers'].cumsum()
df['FN'] = df['Good Borrowers'].sum() - df['Good Borrowers'].cumsum()
df['FP'] = df['Bad Borrowers'].cumsum()
df['TN'] = df['Bad Borrowers'].sum() - df['Bad Borrowers'].cumsum()

df['fpr'] = df.FP / (df.FP + df.TN)
df['tpr'] = df.TP / (df.TP + df.FN)
df['acc'] = (df.TP + df.TN) / total_population

# Calculate the AUC.
auc = np.trapz(df.tpr,df.fpr)
print("AUC:", auc)

# Calculate the Gini coefficient.
gini = 2*auc - 1
print("Gini coefficient:", gini)

# Plot the ROC curve.
plt.plot(df.fpr, df.tpr)
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title("ROC Curve")
plt.fill_between(df.fpr, df.tpr, alpha = 0.1)
plt.text(0.6, 0.4, f'AUC = {auc:0.3}\n Gini = {gini:0.3}')
plt.show()

Please let me know if you need any clarification. I am pretty new to StackExchange so I'm still getting used to this type of communication skill :)