6
$\begingroup$

I am trying to write my own ML library. For speed reasons I started out writing things in C using BLAS, but then I learned that NumPy and Theano also use BLAS. I am wondering if there are huge speed differences between implementations of ML algorithms in C/Python/Matlab/Octave.

Does anybody have some experience or can provide some data for the comparison? If there is no really good reason to write in pure C, I would rather not.

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
8
$\begingroup$

It depends heavily on the algorithm.

There are several things for which writing code in C won't give you any benefit: matrix operations (dot products, element wise multiplications/applications of functions like sin or so, matrix inversions, QR decompositions, ...) because BLAS or LAPACK is called. This makes it possible to implement lots of algorithms easily.

You will have a tough time to match C's performance though when you need to do stuff like trees or huge graphs, which is the case for e.g. decision trees, KNN or sophisticated graphical models with lots of structure.

Some random thoughts:

  • machine learning algorithms are notoriously hard to debug without a reference implementation; C is much harder to debug than Python.
  • you will get to 90% of the performance of C in some cases with Python, but if you really need to be fast, you will have to stick with C
  • Python is growing quite a big eco system for machine learning with theano and sklearn, it's a good time to join.
Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thank you, that was helpful. I would still love to see some benchmarks though. $\endgroup$
    – sjm.majewski
    Commented Jul 29, 2012 at 10:59
  • $\begingroup$ This page discusses some approaches to speeding up python and also provides benchmarks. It's not ML, but it should give you a rough idea of what's possible. $\endgroup$
    – alto
    Commented Jul 29, 2012 at 13:53
  • $\begingroup$ See also scipy-lectures on optimizing $\endgroup$
    – denis
    Commented Sep 10, 2013 at 10:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.