2
$\begingroup$

So I have no calculus experience what so ever and I've been tasked to build a neural network so finding the derivatives is proving quite problematic with my limited calculus experience. I've got the feed forward part working but for the back prop part I need to work out how to calculate the derivatives so far I've come up with this function here

double errorWeight(double errorTotal, double neuronOutput, double sumOfWeightsInNeuron, double weight) {
    return (sumOfWeightsInNeuron / weight) * (neuronOutput / sumOfWeightsInNeuron) * (errorTotal / neuronOutput);
}
  • errorTotal is all the errors from each trainning example added together
  • neuronOutput is the output of the neuron
  • sumOfWeightsInNeuron just sums the weights for that layer together if had 5 neurons going into 2 then it would sum the value of them 10
  • weights together. weight is just the weight we are tring to find the derivertive of with respect to the others.

Is my logic correct? If not where have I gone wrong?

I've initialized my weight to random numbers between 0-1 when I put the first 2 pieces of data through the network i get this result.

error total: 5.19475e+06
deriv 7.31983e+06

Now I attempted to find the derivative with respect to the last weight.

  1. Does this look correct?
  2. would I do this for all the weights in the network?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.