I am looking for a general closed-form formula for the derivatives of a Feed-forward Network with respect to the inputs.
Mathematically, we can write:
$$ \mathbf{y} = f_{FF}(\mathbf{x}) = \mathbf{W}_{h} (\sigma_{h-1}(\mathbf{W}_{h-1} (\ldots \sigma_0(\mathbf{W}_0 \mathbf{x} + \mathbf{b}_0) \ldots) + \mathbf{b}_{h-1})) + \mathbf{b}_{h} $$
Assuming all the activation functions being identical and equal to the $\tanh$, that is $\sigma_h(\cdot) = \tanh(\cdot)$, is there a general closed-formula for the $k$-th order derivatives of $\mathbf{y} = [y_0, \ldots, y_j]$ with respect to $\mathbf{x} = [x_0, \ldots, x_i]$?:
$$ \frac{\partial^k \mathbf{y}^i}{\partial {x^i}^k} = ? $$
This question is a generalization of Neural network derivative with respect to input