3
$\begingroup$

This is a question regarding the specific application of online learning theory in the sense of http://www.mit.edu/~9.520/spring08/Classes/online_learning_2008.pdf

I went through ICML papers for 2017 on online learning, and I can say with confidence that 90% of these papers are theoretical papers and should have most appropriately belonged to a statistics or optimization journal (if weren't for the glorious title of being a Machine Learning researcher).

This field is a mathematician's heaven, but as an engineer, it is hard for me to see how I could use these algorithms in practice. The "applications" that people have came up in this field so far are purely academic and does not clearly show practical real world applications as compared to supervised learning/neural nets for things like optical character recognition, or reinforcement learning for robotics applications. I know the latter works, because there has been much written about them, but I am not convinced that the same is true for online learning algorithms.

For example, all from 2017:

http://proceedings.mlr.press/v70/shamir17a/shamir17a.pdf

http://proceedings.mlr.press/v70/vaswani17a/vaswani17a.pdf

http://proceedings.mlr.press/v70/barmann17a/barmann17a.pdf

http://proceedings.mlr.press/v70/mukkamala17a/mukkamala17a.pdf

All these papers deal with finding some theoretical bound on the "goodness" of their algorithms, but does not clearly show how it relates to practical real world applications.

I am self studying online learning at the moment, using: http://ocobook.cs.princeton.edu/OCObook.pdf

However, I am quickly becoming disillusioned at the lack of serious utility of this mathematical framework. The examples given in the online convex optimization book are very academic. It is difficult to relate them to real life. Even courses on online learning does not provide examples as to how it can be used. I also have tried to dig up some successful applications based on online algorithms, so far I have failed.

Can someone point to some successful real-life applications/implementations of online learning algorithms?

Improve this question
$\endgroup$
5
  • $\begingroup$ Google, Amazon, Netflix... $\endgroup$
    – Tim ♦
    Sep 7, 2017 at 19:18
  • 1
    $\begingroup$ Maybe search for examples of "Bandit algorithms" and "Contextual bandits" and other algorithms that are listed as topics in the course you linked by name. Perhaps they are not categorised in use in industry as "online learning" as per the course, but they are used heavily in web site advertising and content placement for instance. The goals of the bandit algorithms are interesting in their own right - how to make exploratory actions when mistakes made whilst learning have a cost. $\endgroup$
    – Neil Slater
    Sep 7, 2017 at 19:25
  • $\begingroup$ I can definitely agree that Contextual bandits are useful, but this is a typical topic in reinforcement learning, which is already useful by itself. I am more referring to things like FTRL, online gradient descent, online mirror descent, regret minimization. These topics constitute 99% of all online learning algorithms. For example, almost the entire OCO book devote itself to these topics. $\endgroup$
    – Norman
    Sep 7, 2017 at 19:28
  • 1
    $\begingroup$ @DetectiveMooch please define exactly what do you consider as "online learning" so people do not have to go to external resources to understand your question. Otherwise you are likely to receive answers that are not relevant to what you wanted to ask. $\endgroup$
    – Tim ♦
    Sep 7, 2017 at 19:32
  • 1
    $\begingroup$ @DetectiveMooch so please define what exactly your question is about. In many sources reinforced learning would be considered as a kind of online learning. $\endgroup$
    – Tim ♦
    Sep 7, 2017 at 19:41

1 Answer 1

Reset to default
5
$\begingroup$

The UWash page you list gives a few examples right at the top:

Weather forecasting, predicting stock market trends, and deciding which ads to present on a web page are examples of sequential prediction problems

Weather forecasting is, for example, a classic usage of online learning. Any time you want to track a moving object whose location you measure with some error is another canonical example, eg. airplane traffic control. Really any kind of forecasting where information arrives sequentially and your underlying state is changing or developing is a potential application: epidemiology, forest fire tracking, etc. are all areas where online learning is useful.

Improve this answer
$\endgroup$
4
  • $\begingroup$ The OP's question is a bit odd, because it seems very difficult to doubt the utility of online learning. But I think they were very specifically asking for concrete examples of applications of the many new "online algorithms" being used in practice. This answer seems to be saying "they naturally apply to this problem"...but as far as I know those cases really solve the optimization problem offline! $\endgroup$
    – Cliff AB
    Sep 7, 2017 at 19:33
  • 3
    $\begingroup$ "Any time you want to track a moving object whose location you measure with some error is another canonical example" - that's way too broad of a definition. that way you'd say any missile targeting is online learning, that's not true. if your "coefficients" are not updated in real time it's not online. for instance, $y_t=c+\phi y_{t-1}+e_t$ would fall into your definition, and it's clearly not a learning at all, as well as a whole field of error correction models. they "correct" for errors online, aren't they? but they're not learning $\endgroup$
    – Aksakal
    Sep 7, 2017 at 19:34
  • 1
    $\begingroup$ @CliffAB, OP's asking a reasonable question. There's a lot of learning apps, and usually they're offline, in that they're estimated/tuned by developers, then when deployed they don't learn anything, they apply the knowledge that was learned during development. so it's not that often when online learning is used in practice $\endgroup$
    – Aksakal
    Sep 7, 2017 at 19:38
  • $\begingroup$ @Aksakal: my point is that I don't think online learning is a topic that is purely an academic topic with no real applications. Rather, it seems quite the opposite: lots of people in applied situations really want this, but it's a valid question if in practice we can get something much better than, say, batch updates. $\endgroup$
    – Cliff AB
    Sep 7, 2017 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.