I am writing an MLP.

In this book one can find description of the back-propagation learning method.

Starting with the feed-forward ANN one changes weights during learning according to the so called delta-rule (or generalized delta-rule):

$$ \Delta^{p} w_{jk} = -\gamma\frac{\partial E^p}{\partial s^{p}_{k}}y^{p}_{j},\tag{*} $$ where

  • $p$ - current learning pattern
  • $w_{jk}$ - weight between $j$-th and $k$-th nodes
  • $\gamma$ - learning rate
  • $E^{p}$ - current (w.r.t. pattern $p$) squared error
  • $s^{p}_k$ - current (w.r.t. pattern $p$) value of $k$-th node
  • $y^{p}_j$ - current (w.r.t. pattern $p$) output of $j$-th node

In the above mentioned book one can read (p. 37) enter image description here


How do I actually calculate the total weight changes?

The following formula $$ \Delta w_{jk} = \sum_{p=1}^{N_p}\Delta^{p}w_{jk}. $$ does not seem to be correct to me. It's clear if one considers two identical patterns. Instead I use the following formula.

$$ \Delta w_{jk} = \sum_{p=1}^{N_p}a_p\Delta^{p}w_{jk}, $$ where $a_p$ represents "importance" of the current pattern in terms of error: $$ a_p = \frac{E^p}{\sum_p E^p} $$

So what is a correct way to calculate total weight changes?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.