5
$\begingroup$

I am dealing with the nnet package in R. I know that the momentum $\alpha$ is used to decrease the fluctuations in weight changes over consecutive iterations. The original update of the weights states that:

$$\omega_i(t+1) = \omega_i - \eta\frac{\partial E}{\partial w_i},$$ where $E({\bf w})$ is the error function, ${\bf w}$ - the vector of weights and $\eta$ - learning rate.

The weight decay $\lambda$ penalizes the weight changes in the following way:

$$\omega_i(t+1) = \omega_i - \eta\frac{\partial E}{\partial w_i} - \lambda\eta\omega_i$$

Now, what I see on the nnet package is that I can control the $\lambda$ through the parameter "decay". How can I control the learning rate ($\eta$) instead?

$\endgroup$

closed as off-topic by Sycorax, Michael Chernick, Juho Kokkala, kjetil b halvorsen, COOLSerdash Jul 1 '18 at 15:13

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Sycorax, Michael Chernick, Juho Kokkala, kjetil b halvorsen, COOLSerdash
If this question can be reworded to fit the rules in the help center, please edit the question.

Browse other questions tagged or ask your own question.