I've noticed that for sin() like data, I need to use decay which is available in nnet to get the ANN to perform. Why would that be in theory? Also when I run runNN(0.02) over and over, sometimes the model performs beautifully (relative rmse of 0.04) and other times poorly (relative rmse of 0.42). How can one make the training of the nnet not so dependent on the random initialization of the weights?

while(runNN(0.02)>0.1){} # This works but it seems sloppy.

Is there a way to set the training parameters better in order to always converge on the "best" result?

For the training data: Imagine a restaurant has the most customers at noon and the qty of the tips is highest at noon and we want to make a neural net predict tips gained in an hour.

runNN = function(decayParam) {  
  data = data.frame(h=1:24); 
  data$qty <- sin(data$h/48*2*pi)*1000
  data$v = c(paste("actual",decayParam))
      mynn <- nnet(qty~h,data,size=2,decay = decayParam,linout = TRUE,maxit=2000) 
      pred = data.frame(h=1:24); 
      ps <- predict(mynn,pred);
      pred$qty = ps[,1];
  pred$v = c(paste("pred",decayParam))
      rsd = sqrt(mean((data$qty-pred$qty)^2))/mean(data$qty)
  rsds = paste("rsd=",rsd)
  alldata <- rbind(data,pred)
  print(ggplot(data=alldata, aes(x=h, y=qty, group=v, color=v)) + 
          geom_line() + 
print(runNN(0))    # with no decay
print(runNN(0.02)) # with decay

runNN(0)     # with no decay - tends to converge to the population mean
runNN(0.02)  # with decay - works better, but if you run this multiple times, 
             #  sometimes it converges badly.

1 Answer 1


What I ended up doing is having a while loop repeatably create NNs of a small size looking for the best root mean square error.

Essentially: For hidden layer size of 1 to 10 do For trials of 1 to 10: Create NN and test for rmse with test data. Remember NN and best rmse for particular hidden layer size.

Review results choosing the smallest hidden layer size which had a small rmse.

By separating the data into test and train sets, this way you don't pick an over fit NN.

  • $\begingroup$ The real issue is that a learning rate of 0.02 is far too large! $\endgroup$
    – Chris
    Aug 23, 2016 at 12:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.