Imagine a training set with 10 input features. The first two are mass (M) and accelertion (A) and the rest are all useless or irrelevant. The output is Force (F) which of course is equal to M*A.
It seems to me that an ANN architecture has no easy way to just multiply these two features together and give the answer.
If the input vector was [1,1,0,0,0,0,0,0,0,0] and the weights were all initialized to 1, then you would get (1*M) + (1*A). This doesn't provide the necessary straightforward multiplication.
The ANN will obviously figure out the mapping somehow, but will it always do so in a very over-complicated way?
Is this a limitation in some respects or am I misunderstanding something important about ANN's?