25
$\begingroup$

I have been working with large data sets lately and found a lot of papers of streaming methods. To name a few:

However, I have been unable to find any documentation regarding how they compare to each other. Every article I read seem to run experiments on different data set.

I know about sofia-ml, vowpal wabbit, but they seem to implement very few methods, compared to the huge amount of existing methods!

Are the less common algorithms not performant enough? Is there any paper trying to review as many methods as possible?

Improve this question
$\endgroup$
5
  • 7
    $\begingroup$ If there isn't, you should write it yourself :) $\endgroup$
    – Chris C
    Commented Sep 21, 2015 at 12:07
  • 1
    $\begingroup$ you do understand that people in academia have to write papers/come up with new algorithms, and they will search for the data sets on which their algorithm performs best on. I would recommend you just make sure you understand how one library such as vowpal-wabbit runs (ie all parameters etc). $\endgroup$
    – seanv507
    Commented Oct 1, 2015 at 12:18
  • 1
    $\begingroup$ That's actually the opposite! I understood that people chose the best data set and are generally relatively silent on how they cross-validated the algorithms (both theirs and the competing methods). I am rather looking for a streaming version of jmlr.org/papers/volume15/delgado14a/delgado14a.pdf $\endgroup$
    – RUser4512
    Commented Oct 1, 2015 at 12:21
  • 1
    $\begingroup$ Really like the JMLR paper you linked. I myself so not know a similar comparison for streaming algorithms. Probably because streaming is more niche and also because while it is already hard to compare classifiers for static datasets it is even more complicated to make a fair comparison for streaming data. $\endgroup$
    – Steffen Moritz
    Commented Nov 16, 2016 at 19:43
  • 1
    $\begingroup$ Although these do not specifically answer your question, two related resources are: Evaluating Algorithms that Learn from Data Streams by Gama et al., which discusses evaluation techniques, and MOA (Massive Online Analysis), an open source framework for data stream mining which incorporates the ability to evaluate performance. $\endgroup$
    – user77876
    Commented Aug 22, 2017 at 14:40

1 Answer 1

Reset to default
2
$\begingroup$

A rigorous survey of multiple algorithms similar to the Delgado paper you linked is not available as far as I know, but there have been efforts to gather results for families of algorithms.

Here are some sources I find useful (disclaimer: I publish in the area, so it's likely I'm biased in my selection):

Some sofware packages:

I can add more info and sources if needed. As others have said the field could use a comprehensive survey.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.