To me, this is a (provocative, and intentionally so) restatement of the universal approximation theorems. Loosely speaking, the universal approximation theorems say that, given enough parameters, a neural network can get as close to a decent function as you want.
In this context, it means that you can let the neural network figure the composition of the feature extraction function and the regression function.
However, if you have insights into the features that would be important, you can give your neural network a break and do some of the work for it. Why make the network figure out that quadratic terms are important when domain knowledge (physics, chemistry, biology, etc) says that you know the quadratic term is important? You can, perhaps, use fewer parameters and put yourself at a lower risk of overfitting.