I hope this isn't a stupid question. I'm trying to reduce overfitting on my perceptron network by adding in a regularization term. However I am not sure where the actual term goes...

Usually the regularization term is shown in the literature as being added to the cost function, like so:

$$J(w) = \sum_{i=1}^n (y^i - ypr^i)^2 + \lambda\sum_{j = 1}^m|w_j|$$

However I don't know how exactly this applies to the weight update equation for the perceptron:

$$w = w_j(t) + n(y^i - ypr^i)x$$

Do I put the regularization term within the brackets (above) and therefore multiply it by the learning rate and the data?...

Many thanks in advance


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.