# Does online learning theory have any real world applications?

This is a question regarding the specific application of online learning theory in the sense of http://www.mit.edu/~9.520/spring08/Classes/online_learning_2008.pdf

I went through ICML papers for 2017 on online learning, and I can say with confidence that 90% of these papers are theoretical papers and should have most appropriately belonged to a statistics or optimization journal (if weren't for the glorious title of being a Machine Learning researcher).

This field is a mathematician's heaven, but as an engineer, it is hard for me to see how I could use these algorithms in practice. The "applications" that people have came up in this field so far are purely academic and does not clearly show practical real world applications as compared to supervised learning/neural nets for things like optical character recognition, or reinforcement learning for robotics applications. I know the latter works, because there has been much written about them, but I am not convinced that the same is true for online learning algorithms.

For example, all from 2017:

http://proceedings.mlr.press/v70/shamir17a/shamir17a.pdf

http://proceedings.mlr.press/v70/vaswani17a/vaswani17a.pdf

http://proceedings.mlr.press/v70/barmann17a/barmann17a.pdf

http://proceedings.mlr.press/v70/mukkamala17a/mukkamala17a.pdf

All these papers deal with finding some theoretical bound on the "goodness" of their algorithms, but does not clearly show how it relates to practical real world applications.

I am self studying online learning at the moment, using: http://ocobook.cs.princeton.edu/OCObook.pdf

However, I am quickly becoming disillusioned at the lack of serious utility of this mathematical framework. The examples given in the online convex optimization book are very academic. It is difficult to relate them to real life. Even courses on online learning does not provide examples as to how it can be used. I also have tried to dig up some successful applications based on online algorithms, so far I have failed.

Can someone point to some successful real-life applications/implementations of online learning algorithms?

– Tim
Sep 7 '17 at 19:18
• Maybe search for examples of "Bandit algorithms" and "Contextual bandits" and other algorithms that are listed as topics in the course you linked by name. Perhaps they are not categorised in use in industry as "online learning" as per the course, but they are used heavily in web site advertising and content placement for instance. The goals of the bandit algorithms are interesting in their own right - how to make exploratory actions when mistakes made whilst learning have a cost. Sep 7 '17 at 19:25
• I can definitely agree that Contextual bandits are useful, but this is a typical topic in reinforcement learning, which is already useful by itself. I am more referring to things like FTRL, online gradient descent, online mirror descent, regret minimization. These topics constitute 99% of all online learning algorithms. For example, almost the entire OCO book devote itself to these topics. Sep 7 '17 at 19:28
• @DetectiveMooch please define exactly what do you consider as "online learning" so people do not have to go to external resources to understand your question. Otherwise you are likely to receive answers that are not relevant to what you wanted to ask.
– Tim
Sep 7 '17 at 19:32
• @DetectiveMooch so please define what exactly your question is about. In many sources reinforced learning would be considered as a kind of online learning.
– Tim
Sep 7 '17 at 19:41

• "Any time you want to track a moving object whose location you measure with some error is another canonical example" - that's way too broad of a definition. that way you'd say any missile targeting is online learning, that's not true. if your "coefficients" are not updated in real time it's not online. for instance, $y_t=c+\phi y_{t-1}+e_t$ would fall into your definition, and it's clearly not a learning at all, as well as a whole field of error correction models. they "correct" for errors online, aren't they? but they're not learning Sep 7 '17 at 19:34