1
$\begingroup$

I'm new to data science, and am working on projects to develop my skills. I'm currently working on the Loan Prediction practice competition offered by Analytics Vidhya.The goal is to build a predictive model that will determine whether a loan will be approved or not.

So far, my models (logistic regression, random forest) do not perform any better than simply using the applicant's credit history as a predictor.

I've read through the provided tutorial (I don't have high enough rep to include all the links): https://www.analyticsvidhya.com/blog/2016/01/complete-tutorial-learn-data-science-python-scratch-2/ which details the same results. The tutorial mentions that feature engineering will be important for this (and most) projects. I've found a few other resources for feature engineering ( http://data-informed.com/how-to-improve-machine-learning-tricks-and-tips-for-feature-engineering/ and http://adataanalyst.com/machine-learning/comprehensive-guide-feature-engineering/ ) being the most helpful) but I'm still unable to build a model with higher accuracy.

The variables included in the data set are:

Variable          Description
-------------     -----------------
Loan_ID           Unique Loan ID
Gender            Male/ Female
Married           Applicant married (Y/N)
Dependents        Number of dependents
Education         Applicant Education (Graduate/ Under Graduate)
Self_Employed     Self employed (Y/N)
ApplicantIncome   Applicant income
CoapplicantIncome Coapplicant income
LoanAmount        Loan amount in thousands
Loan_Amount_Term  Term of loan in months
Credit_History    credit history meets guidelines
Property_Area     Urban/Semi Urban/Rural
Loan_Status       Loan approved (Y/N)

I've first multiplied LoanAmount by 1000, to have the value in dollars.

I've tried creating the following new features:

TotalIncome = ApplicantIncome + CoapplicantIncome

Monthly_Payment = LoanAmount / Loan_Amount_Term

Payment_Ratio = Monthly_Payment / TotalIncome

household_size = the number of people in the household, derived from marital status and number of dependents

I also found that many of the continuous variables (LoanAmount, TotalIncome, Payment_Ratio) are right-skewed and have transformed them by taking the log.

None of these changes have had any effect. It seems that no matter what variables I include in my model, it will not achieve better accuracy than the model that only considers Credit_History.

Are there any obvious features I've overlooked?

What should I try next?

Improve this question
$\endgroup$
2
  • $\begingroup$ Did you try interactions? $\endgroup$
    – kjetil b halvorsen ♦
    Commented May 19, 2017 at 18:08
  • $\begingroup$ No. I'll look at that and update my question. $\endgroup$
    – Richard Kublik
    Commented May 19, 2017 at 18:10

1 Answer 1

Reset to default
0
$\begingroup$

People usually start feature engineering with different basis expansions. Such as polynomial expansion, splines, Fourier basis expansion etc.

Here are some related posts

What's wrong to fit periodic data with polynomials?

Why I am getting different predictions for manual polynomial expansion and using the R `poly` function?

Also, neural network can do the feature engineering for us.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'm not sure how to apply this, which variables do you suggest I use a basis expansion for? $\endgroup$
    – Richard Kublik
    Commented May 19, 2017 at 20:11
  • $\begingroup$ @RichardKublik if you want "accurate" model not "interpretive" model, expansion can be used in all variables. to avoid overfitting, regularization can be used. $\endgroup$
    – Haitao Du
    Commented May 19, 2017 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.