0
$\begingroup$

If I have understood this correctly, a result from Hornik et al.'s Universal Approximation of an Unknown Mapping and Its Derivatives Using Multilayer Feedforward Networks essentially states that, if we have a function $F\in C^n$ and a neural network $F_{ANN}$ that approximates $F$, then, if an activation function that is of class $C^n$ is used, then $F_{ANN}$ arbitrarily approximates $F$ and all its derivatives up to order $n$.

($F$ is said to be of class $C^{n}$ if its first through to $n$ derivatives are continuous).

If, for example, we have a neural network with 3 hidden layers, each using an activation function that is of class $C^n$, and an output layer that simply uses the linear (identity) activation function, does the network still approximate $F$ and all of its derivatives up to order $n$?

My thinking is that because the output layer activation is just the linear activation function, the answer is yes. But I am not sure, as the activation function used in the hidden layers (which is of class $C^n$) is not used on the output layer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.