2
$\begingroup$

I'm trying to understand this notation:

enter image description here

Specifically, what does the summation compute at each iteration? Looks like for each training sample, I compute the error (difference between target and output for that sample) and multiple that by each term in that same sample. Is this a nested summation?

Example data:

// Samples
[
  [1, 2, 3],
  [4, 5, 6],
]

// Predicted output
[
  [23],
  [42],
]

// Target output
[
  [15],
  [16],
]

What does that summation expand into?

$\endgroup$
1
  • $\begingroup$ No, it's not a nested summation. You get out a vector with one element for each $j$ - each entry in the vector corresponds to one $\Delta w_j$. $\endgroup$
    – jbowman
    Jul 6, 2017 at 16:01

1 Answer 1

1
$\begingroup$

As far as I understand, you first compute the difference between target and output such as

$E_j = \sum_{i} target_{ji} - output_{ji}$

Here

$E_1 = 15 - 23 = -8$ and $E_2 = 16 - 42 = -26$

Now for each weight, you will take into account the input dimension associated with the weight thus:

$\Delta w_1 = \eta * ( (-8 * 1) + (-26 * 4))$

$\Delta w_2 = \eta * ( (-8 * 2) + (-26 * 5))$

$\Delta w_2 = \eta * ( (-8 * 3) + (-26 * 6))$

So the $x_{ij}$ in the formula is the data dimension $j$ for sample $i$.

We might prefer though to compute everything much more compact such as:

$\vec{\Delta w} = \eta * \vec{E} \times \vec{x}$, where $\vec{E}$ is an (1xn) array and $\vec{x}$ is the input data such as (nx3).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.