I've heard neural networks be defined as computing a function but if the activation functions are already chosen for the hidden layers and output layers, and the cost function is also chosen, then isn't the overall function already determined and the neural network is just trying to find the parameters of that function that determine the best decision boundary?
In Goodfellow's DL book it says:
"If we use a sufficiently powerful neural network, we can think of the neuural network as being able to represent any function from a wide classs of functions, with this class being limited only by features such as continuity and boundedness rather than by having a specific parametric form. From this point of view we can view the cost function as being a functional rather than just a function."
Is what he means here that the function is essentially limitless and is so only really determined by its parameters, ergo, finding the parameters is defining the function?