I trained a machine learning regressor for a target variable: loan charged off amount.
My task is to predict the charged off amount on a future date given a bunch of loan data as input. So I choose to use regression method.
My dataset comprises two types of variables:
- static variables: like loan amount, term, borrower age, borrower employment type etc.
- Dynamic variables:the loan age in days on observation (e.g. I observed a loan at its 37 days old and I can see after 30 days it charged off $10000. So I record the loan age in days on observation as 37.), incremental days (e.g. 30 days in the example), principal remaining on observation day, the days remaining till the end of the loan (say the loan 1827 days in activeness by contract (5 years) and it had spent 827 days and 1000 days remaining. )
The prediction results were shown as below. Each point represented a cohort of a quarter (e.g. 2014-07-01 indicates loans whose loan contract date were between 2014-07-01 and 2014-09-30). The results were about the predicted loss rate after 90 days. It seemed only those very young loans after 2017-01-01 would be predicted with some loss and others not.
My dataset had imbalanced value in the target variable 'Charged Off Amount', i.e. out of about 28000 loans only 1200 loans had non-zero charged off amount and others with zero.
I know I can't present the whole project with every detail here so sorry for confusion on some points.
I just wanna ask for help on a more general issue: In regression modelling, How can I design my dataset to train a model which can behave as expected. Like in my example, I wanna see after 90 days, the curve should move upwards to an reasonable extent rather than what we can see that for most of the cohort it nearly don't move a bit.
In other words, if you get a predicted result which disobey the empirical vision and then what could one do to guide the model to the track which makes more sense in realistic application.
Welcome anyone to advice how to improve my question or some empirical skills to finish this task. I just try my best to present my question here and if not clear enough please suggest and help.