# Issues with neural network

I am having some issues with using neural network. I am using a non linear activation function for the hidden layer and a linear function for the output layer. Adding more neurons in the hidden layer should have increased the capability of the NN and made it fit to the training data more/have less error on training data.

However, I am seeing a different phenomena. Adding more neurons is decreasing the accuracy of the neural network even on the training set.

Here is the graph of the mean absolute error with increasing number of neurons. The accuracy on the training data is decreasing. What could be the cause of this?

Is it that the nntool that I am using of matlab splits the data randomly into training,test and validation set for checking generalization instead of using cross validation.

Also I could see lots of -ve output values adding neurons while my targets are supposed to be positives. Could it be another issues?

I am not able to explain the behavior of NN here. Any suggestions?

Here is my code

targets = dayofyear_targets(:, i+1);
net = newfit(train_data', targets', 4);
net = init(net);
net.performFcn = 'mae';
net.layers{2}.transferFcn = 'purelin';
net.trainParam.max_fail = 10;
net.layers{1}.transferFcn = 'tansig';
net = train(net, train_data', targets');
results = sim(net, train_data')';
diff = abs(results-targets);
mae = sum(diff(:))/ num_samples

• Are you sure that this plot represents training set errors, but not validation or test set? Check that point – O_Devinyak Sep 4 '13 at 17:54
• @O_Devinyak. Yes I am sure it's training set errors – user34790 Sep 4 '13 at 18:03
• Than I have no glue besides possible errors in code. Maybe your neural network is performing standardization, so you are comparing standardized output with non-standardized actual values? – O_Devinyak Sep 4 '13 at 18:27
• @O_Devinyak. I have added my code. Yeah, the NN tool is performing minmax normalization to make the output in the range [-1 1]. However, I am using nn's sim function to predict and compare it with the actual values. So I guess it should have been fine – user34790 Sep 4 '13 at 18:34
• Hm. I don't see any flaws (besides redundant quotation marks - are they placed intentionally?). Maybe it is worth to try the same with default activation functions. – O_Devinyak Sep 4 '13 at 19:04