Problem using MLP function from package RSNNS in R

Question

im trying to use a MLP to estimate a non-linear function, but for some reason, the ANN is giving me weights that make every input a single value of output. Looks like its estimating by just a constant. In fact this is happening even for very simple examples, like a linear regression with white noise. RBF works just fine every time.

I know that for a linear regression i could use some analitical approach or simply the lms algorithm, this is just a example where MLP should work but im doing something wrong. I tried different activation functions, learning parameters, number of hidden layers and so on. I also tried with different polynomials and MLP always returns me a weighed matrix that converges to a constant near the unconditional mean of the training data.

library(RSNNS)  
set.seed(1)  
e<-rnorm(500,sd=100)

x<-seq(1:500)

y<-2*x+e

model_mlp<-mlp(x,y,size=c(1),maxit=1000,initFunc="Randomize_Weights",initFuncParams=c(-0.1,0.1),learnFunc="Std_Backpropagation",learnFuncParams=c(0.1,0),hiddenActFunc="Act_Logistic",linOut=TRUE)

predictions_mlp<-predict(model_mlp,t(t(x)))  
plot(predictions_mlp,type="l")  
lines(y,col="2")

model_rbf<-rbf(x, y, size = c(50), maxit = 1000,initFunc = "RBF_Weights", initFuncParams = c(0, 1, 0, 0.02, 0.04),learnFunc = "RadialBasisLearning", learnFuncParams = c(1e-05, 0, 1e-05,0.1, 0.8), linOut = TRUE)

predictions_rbf<-predict(model_rbf,t(t(x)))  
plot(predictions_rbf,type="l")  
lines(y,col="2")    

I couldnt find this doubt in another topic, sorry if its repeated and sorry about the english.

thanks!

Answer 1

model_mlp<-mlp(x,y,size=c(1),maxit=1000,learnFunc="Rprop",linOut=TRUE)
predictions_mlp<-predict(model_mlp,t(t(x)))   # now gives 500 unique predictions

Backpropagation seems extremely sensitive to the learning parameters while Rprop doesn't. If you still want to use backpropagation you should play with your learning rates more.