0
$\begingroup$

Preliminaries

I am writing an MLP.

In this book one can find description of the back-propagation learning method.

Starting with the feed-forward ANN one changes weights during learning according to the so called delta-rule (or generalized delta-rule):

$$ \Delta^{p} w_{jk} = -\gamma\frac{\partial E^p}{\partial s^{p}_{k}}y^{p}_{j},\tag{*} $$ where

  • $p$ - current learning pattern
  • $w_{jk}$ - weight between $j$-th and $k$-th nodes
  • $\gamma$ - learning rate
  • $E^{p}$ - current (w.r.t. pattern $p$) squared error
  • $s^{p}_k$ - current (w.r.t. pattern $p$) value of $k$-th node
  • $y^{p}_j$ - current (w.r.t. pattern $p$) output of $j$-th node

In the above mentioned book one can read (p. 37) enter image description here

Question

How do I actually calculate the total weight changes?

The following formula $$ \Delta w_{jk} = \sum_{p=1}^{N_p}\Delta^{p}w_{jk}. $$ does not seem to be correct to me. It's clear if one considers two identical patterns. Instead I use the following formula.

$$ \Delta w_{jk} = \sum_{p=1}^{N_p}a_p\Delta^{p}w_{jk}, $$ where $a_p$ represents "importance" of the current pattern in terms of error: $$ a_p = \frac{E^p}{\sum_p E^p} $$

So what is a correct way to calculate total weight changes?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.