Let's say we want to do regression for simple
f = x * y using standard fully-connected deep neural network. The network takes
y as input, and should learn to output
x * y.
I remember that there is research proving that NN with one hidden layer can approximate any function, but I have tried and was unable to approximate even this simple multiplication.
Only log-normalization of the data helped
m = x*y => ln(m) = ln(x) + ln(y) because it turns multiplication into addition, but that looks like a cheat. Can NN do this without log-normalization?
The answer seems to be Yes, so the question is more what should the type/configuration/layout of such NN be?