So I have no calculus experience what so ever and I've been tasked to build a neural network so finding the derivatives is proving quite problematic with my limited calculus experience. I've got the feed forward part working but for the back prop part I need to work out how to calculate the derivatives so far I've come up with this function here

double errorWeight(double errorTotal, double neuronOutput, double sumOfWeightsInNeuron, double weight) {
    return (sumOfWeightsInNeuron / weight) * (neuronOutput / sumOfWeightsInNeuron) * (errorTotal / neuronOutput);
  • errorTotal is all the errors from each trainning example added together
  • neuronOutput is the output of the neuron
  • sumOfWeightsInNeuron just sums the weights for that layer together if had 5 neurons going into 2 then it would sum the value of them 10
  • weights together. weight is just the weight we are tring to find the derivertive of with respect to the others.

Is my logic correct? If not where have I gone wrong?

I've initialized my weight to random numbers between 0-1 when I put the first 2 pieces of data through the network i get this result.

error total: 5.19475e+06
deriv 7.31983e+06

Now I attempted to find the derivative with respect to the last weight.

  1. Does this look correct?
  2. would I do this for all the weights in the network?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.