25
$\begingroup$

I have been working with large data sets lately and found a lot of papers of streaming methods. To name a few:

However, I have been unable to find any documentation regarding how they compare to each other. Every article I read seem to run experiments on different data set.

I know about sofia-ml, vowpal wabbit, but they seem to implement very few methods, compared to the huge amount of existing methods!

Are the less common algorithms not performant enough? Is there any paper trying to review as many methods as possible?

$\endgroup$
  • 7
    $\begingroup$ If there isn't, you should write it yourself :) $\endgroup$ – Chris C Sep 21 '15 at 12:07
  • 1
    $\begingroup$ you do understand that people in academia have to write papers/come up with new algorithms, and they will search for the data sets on which their algorithm performs best on. I would recommend you just make sure you understand how one library such as vowpal-wabbit runs (ie all parameters etc). $\endgroup$ – seanv507 Oct 1 '15 at 12:18
  • 1
    $\begingroup$ That's actually the opposite! I understood that people chose the best data set and are generally relatively silent on how they cross-validated the algorithms (both theirs and the competing methods). I am rather looking for a streaming version of jmlr.org/papers/volume15/delgado14a/delgado14a.pdf $\endgroup$ – RUser4512 Oct 1 '15 at 12:21
  • 1
    $\begingroup$ Really like the JMLR paper you linked. I myself so not know a similar comparison for streaming algorithms. Probably because streaming is more niche and also because while it is already hard to compare classifiers for static datasets it is even more complicated to make a fair comparison for streaming data. $\endgroup$ – stats0007 Nov 16 '16 at 19:43
  • 1
    $\begingroup$ Although these do not specifically answer your question, two related resources are: Evaluating Algorithms that Learn from Data Streams by Gama et al., which discusses evaluation techniques, and MOA (Massive Online Analysis), an open source framework for data stream mining which incorporates the ability to evaluate performance. $\endgroup$ – user77876 Aug 22 '17 at 14:40
1
$\begingroup$

A rigorous survey of multiple algorithms similar to the Delgado paper you linked is not available as far as I know, but there have been efforts to gather results for families of algorithms.

Here are some sources I find useful (disclaimer: I publish in the area, so it's likely I'm biased in my selection):

Some sofware packages:

I can add more info and sources if needed. As others have said the field could use a comprehensive survey.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.