I am new to Machine Learning and learned about Decision Tree Learning as part of a problem I am tackling on for an Undergraduate Research Assistant position.

The problem: From a set of positional data of hockey players and pucks, how can one recognize when a shot is made?

The issue: I have access to two games of hockey, one I will use to train my decision tree and the other I will use to test. However, for each game of hockey, we have about 80000 frames of data but only about 15 of them are annotated as shots made.

My question: Since we have a lot of negative frames but comparatively very little positive frames, will a decision tree model lead to overfitting? If yes, are there any alternate solutions?


Over-fitting happens when you have decision boundaries that are too complex and try to accommodate every single training example at the expense of generalization performance. Trees need to be pruned so that they do not over-fit. Most software packages will do that automatically for you. Over fitting should not be you primary concern.

With such extreme class imbalance, you first concern should be the sensitivity vs. specificity trade-off. This problem might be related to over-fitting in that the automatic pruning might put your few positive cases as minorities in mixed leaves (harming sensitivity). You could react by either disabling the pruning, but then you end up with a model that will only judge frames that are very similar to your positive training frames as positive. Or you could react by lowering the threshold for positive classification (harming specificity).

Another concern should be if your test-set has the same characteristics as the training set. If the second game is played by different teams with different players, they could shoot from very different situations which would prevent generalization.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.