Using a relatively standard MLP neural network, I model the total duration of an activity based on many counts of sub-activities, where the relationships between the sub-activities and duration may be non-linear. I mean-center and z-normalize the inputs for insertion into the network, meaning that some input examples have negative values for some features.
I want to understand what sub-activities have the greatest impact on total activity duration, across the input space. I use sensitivity analysis to do this.
Here, for each input example, I calculate the partial derivative of the response with respect to each feature. However, I get a mixture of positive and negative derivatives!
A negative derivative for an input means that the response (total duration) decreases when we increase the input (number of sub activities). Of course, this makes no sense, because a sub activity always takes a small non-negative amount of time to occur.
Does anyone have any idea why I would sometimes be getting these negative derivatives?