# Methods of preventing overfitting other than adding noise to data?

I am about to begin running a data science competition. I have built a "baseline" model to test how accurate some of the competitors' submissions may be (I've set the target as to what they should aim to beat), and a simple, out-of-the-box decision tree model (sklearn.tree.DecisionTreeClassifier) scored 100% accuracy, even after doing a train-test split.

This is a problem because there's no point in running a full DS challenge if some Average Joe off the street can just fit some black-box to the problem and get 100% accuracy. There needs to be room for error. I'm thinking of adding noise to the dataset (both the dependent and independent variables) to try to throw people off a bit to prevent overfitting. In what other ways, outside of adding noise and creating train and withheld-test datasets, can I modify the data to make this problem less "easy"?

• Rather that modifying your existing data, would it be possible for you to switch to a more difficult problem/dataset?
– mkt
Aug 2 '17 at 17:51
• @mkt For the sake of this question, let's assume that I cannot switch to another dataset. The goal of the challenge is for competitors to flex their mathematical skills -- the problem they're solving isn't necessarily the focus here. I really want to showcase who's great at building mathematical models, not necessarily who's the best data scientist (which would incorporate SME knowledge). Aug 2 '17 at 18:27
• Just to be clear here: This is 100% accuracy on a separate test dataset that would not be available to the participants and is separate from what the decision tree was trained on? If not: accuracy on the training dataset is not really a meaningful goal to look at. If it is on a spearate test dataset: besides picking a different example, one possibility might be to make the training dataset smaller. Aug 2 '17 at 19:48
• @Björn I found a slight error after I posted this; the 100% was on the training set... Bad form, I know. But I think my question still holds: in what other ways can I obfuscate or muddy the data a bit to make the classification task more difficult? Aug 2 '17 at 20:35
• Where's the problem? 100% in the training set is not a problem. Aug 3 '17 at 5:00