Can you know in advance whether machine learning is going to work for a problem?

Question

Machine learning is in fashion these days in many industries. So business folks think it can be applied to anything to make things great again.

My experience tells me that there are problems where it doesn't work at all. Moreover, it seems to me that one can say in advance whether a machine learning going to work or not just by looking at the problem to solve.

Do you have heuristics that tell you whether it's worth trying ML on a problem before even trying?

Here's how I look at it. For instance, I want to build a robot that sorts out red and white balls into separate baskets. This is a great problem for ML because a human can easily distinguish red and white ball. So, a bunch of robots will do this faster.

Another problem: find out whether a person is lying or not. A human cannot do this reliably, so a robot can't do it either.

Basically, if an average human can do the job, then a machine can too.

CLARIFICATION: @Kodiologist refers to an example of a small talk, which an average person can, but machines can't yet. That's a good point. I had to qualify my last statement, of course. I meant the tasks that are mundane in nature, e.g. recognizing a song played on radio vs. writing a song. Machines are better than humans at former, but suck at the latter - and that seems obvious to me.

In recognizing a song the only thing that prevents me from doing it is that I simply haven't heard the song, or maybe I don't remember it. If I remember the song I'll always recognize. So, it must be possible to train a machine to do it.

UPDATE: another set of problems that are good for ML is the ones where there's clearly strong stable relationship between inputs and outputs. For instance, pedometers and activity trackers. In principle we could calculate exactly how the movement of the body translates into accelerometer readings, it's just too cumbersome. So, ML works great.

In contrast predicting the stock market crash is impossible neither for a machine nor for a human, because there's no string and stable relationship to inputs.

Answer 1

Your last statement is definitely not true in general. For example, conversation bots are still not very smart. They are infrequently capable of having small talk, let alone passing a Turing test with a competent judge or answering general knowledge questions. Watson is something, but there's still a long way to go. Identifying what's in a picture is another problem that's far from solved despite getting a lot of attention and some exciting advances.

Getting back to everyday uses of machine learning and statistics, the usual things that can tip you off right out of the gate that you probably won't get anywhere are: not much data being available, not much variability in the IVs, very wide data, etc.