I have a data set where one independent variable perfectly predicts the outcome. Specifically, age will perfectly predict if the user will buy the product.

However, I also have a number of other variables such as income, marital status, and gender.

When creating a model using a decision tree, it only uses the age variable and has 100% accuracy on the testing data.

I also tried logistic regression, which incorporates more variables. Though its accuracy is about 94%.

My question is how I should approach this issue? Should I just not use all of the other variables and only use age? Or should I use a seemingly less accurate model that uses more variables but is more flexible?


Update - some more information about data set.

My data is fictional with a size of 2,000. 950 people bought the product (represented as 1) and 1050 did not buy (represented as 0). My concern is that this problem seems too easy, the objective is to determine which variables should be looked at to determine if someone will buy a product. Seems suspiciously easy that it would only be one (age).

Below is a plot of the age data age vs bought product


migrated from stackoverflow.com Dec 24 '17 at 19:33

This question came from our site for professional and enthusiast programmers.

  • 3
    $\begingroup$ If you are able to perfectly predict with one variable you should be happy. However, I find that unlikely with real data of sufficient size. $\endgroup$ – Roland Dec 24 '17 at 18:58
  • $\begingroup$ Without seeing dataset or code, I am a bit concerned with your use of "accuracy" as a metric for model assessment. Does your training set consist of a balanced 1/0 categories? Is your training set sufficiently large? Perhaps use of sensitivity, specificity, and/or AUC might be more informative. $\endgroup$ – David C. Dec 24 '17 at 19:11
  • $\begingroup$ Agree with @DavidC. on using the AUC as a performance metric. Also, you haven't really provided any details about your dataset. How many data points do you have? Are the classes equally represented in the data? How many classes are there? $\endgroup$ – Sam Dec 24 '17 at 19:16
  • $\begingroup$ @Daruchini Thanks for the response. I added more information in the main post. $\endgroup$ – rawrzors Dec 24 '17 at 19:55
  • 2
    $\begingroup$ Simulate different data. You clearly created a test case with an "effect size" (difference of means of age divided by pooled standard deviation) that resulted in "perfect separation". $\endgroup$ – DWin Dec 24 '17 at 20:07

In the added details you note that your data is "fictional" - by which I assume you mean that you simulated it. The way you simulated the data, only age is needed. But the simulated data is unrealistic in the extreme. In your data set, no one under 50 bought the product and no one over 30 bought it; also, there are no people between 30 and 50! This makes no sense. There are surely some products which are only (or almost only) bought by older people, but in that case, it is usually obvious. Further, there is no product which all older people buy. Also, there is no product I can think of that will be marketed to people between 10 and 30 and between 50 and 80, but not between 30 and 50.

So, you need to simulate something realistic.

Your data is like trying to tell basketball centers from jockeys by height: It's easy, but it's pointless.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.