# Preliminaries

I am writing an MLP.

In this book one can find description of the back-propagation learning method.

Starting with the feed-forward ANN one changes weights during learning according to the so called delta-rule (or generalized delta-rule):

$$\Delta^{p} w_{jk} = -\gamma\frac{\partial E^p}{\partial s^{p}_{k}}y^{p}_{j},\tag{*}$$ where

• $p$ - current learning pattern
• $w_{jk}$ - weight between $j$-th and $k$-th nodes
• $\gamma$ - learning rate
• $E^{p}$ - current (w.r.t. pattern $p$) squared error
• $s^{p}_k$ - current (w.r.t. pattern $p$) value of $k$-th node
• $y^{p}_j$ - current (w.r.t. pattern $p$) output of $j$-th node

In the above mentioned book one can read (p. 37)

# Question

How do I actually calculate the total weight changes?

The following formula $$\Delta w_{jk} = \sum_{p=1}^{N_p}\Delta^{p}w_{jk}.$$ does not seem to be correct to me. It's clear if one considers two identical patterns. Instead I use the following formula.

$$\Delta w_{jk} = \sum_{p=1}^{N_p}a_p\Delta^{p}w_{jk},$$ where $a_p$ represents "importance" of the current pattern in terms of error: $$a_p = \frac{E^p}{\sum_p E^p}$$

So what is a correct way to calculate total weight changes?