How to do Machine Learning in the Right way?

Question

I have a basic understanding about Machine Learning in general. My question is how it is done in the practical applications of it.

If I take the following definition of ML

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

It talks about Experience E. What I understand from the above is the more data you give to the algorithm E increases, and that in turn increases P.

Now let's consider a scenario where you build a decision tree model from 10,000 data rows available. Now I have the model, so can I say that my model has learned and just stop there! (use that model for prediction from that point on-wards forever)?

According to the definition, I need to feed more data so the experience increases, and in turn I get a performance improvement.

So is Machine Learning a continuous process so that You cannot build the model and just stop there. Do we need to feed more data to the algorithm time by time and improve the model so that the model actually LEARNS?

Answer 1

It sounds like the distinction you're actually asking about is online learning vs. batch learning, although this terminology is somewhat overloaded so we have to be careful. There is another sense of online learning where it simply means "showing the algorithm the training examples one-by-one and never more than once." That usage is also legitimate but its not very relevant to your question so it won't be the sense in which I use "online learning" in this answer.

The type of online learning you would be interested in is when an agent continuously learns from a potentially changing environment. The canonical example is the multi-armed bandit problem. Here, an agent is sitting in front of a number of slot machines (one-armed bandits) all with different payoffs and has to decide which lever to pull. In some variations, the payoff of each slot machine can even change over time! The agent has to start by exploring to discover which machine has the highest payoff, then shifts to exploiting what's its learned. However, if the payoffs are changing, it has to continue to spend at least some of its time exploring. Any agent which stops learning in a changing environment is doomed to get stuck in a sub-optimal strategy. There are many potential algorithms for this problem.

In industry, most machine learning models have a very different life cycle: they are trained on some carefully curated training set, tested and validated, then released into production where they never learn again but only spit out responses based on what they learned in their youth. This approach has a lot of benefits too, namely that humans are in the loop and can prevent broken models from being released and try to optimize each model (with hyper-parameter selection, for example) before each release. This works because the environments are changing too slowly for it to be necessary for the model to have learned from the day before. Although sometimes you see an A/B test or HFT algorithm that runs online learning in production.

I wouldn't describe either as "real" or "true" machine learning; batch learning meets the definition of "learning from experience" just as well, and they have different use cases, strengths, and weaknesses.

Answer 2

You state :

"According to the definition, I need to feed more data so the experience increases, and in turn I get a performance improvement."

The definition doesn't state that. There is no where in the definition that says that if you give the algorithm more data that it will necessarily improve performance. It just says that if you give it data to learn from and that the algorithm will have an associated performance.

While it seems intuitively obvious that if give the algorithm more rows of data it will perform better it isn't necessarily the case.

So whats the reality? The terribly frustrating answer is "It depends". Depends on what? Many, many things. Expect to spend years learning that.

If you spend more effort on the model will you get better results? Maybe, maybe not. It depends. There is an awful lot of trying it and see what happens.

Its a case of knowing what your objectives are. For what problem your trying to solve is the model good enough?