4
$\begingroup$

What are some techniques by which we can verify that a current implementation of a machine learning algorithm is correct? Is using the results of a benchmark dataset is enough?

$\endgroup$
3
  • 3
    $\begingroup$ Unit testing is the way to go. Make toy data sets for which you know what to expect and verify that you get that results. $\endgroup$ Mar 22, 2015 at 13:00
  • $\begingroup$ Thanks a lot for the reply, sometimes that's not a feasible solution, for example if I wanted to validate an LDA topic modeling algorithm? $\endgroup$
    – Magellanea
    Mar 22, 2015 at 14:00
  • 1
    $\begingroup$ @Magellanea it's feasible and natural if you're writing good code $\endgroup$ Mar 22, 2015 at 15:31

1 Answer 1

5
$\begingroup$

This question has several right answers. Here are three:

  1. Unit test your code. Rather than testing the entire algorithm, test each step separately to make sure it gives the right result. The test cases only need to be as complicated as the individual step. This will enforce good coding habits anyway.
  2. Use toy data. If you are fitting a SVM, you should be able to find the correct support vector on a linearly separable data set.
  3. Replicate an existing result. Many, if not most, stats and ML method papers include a demonstration or experiments section. Replicate their demo and make sure your results match.

And of course don't forget the obvious: if the results don't make any sense, re-check your code before you re-check your model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.