I am not quite sure if this is the right place for a question like this, but asking anyway. Having <14x10 double> input matrix (manually normalized) and <5x10 double> output matrix (manually normalized), after a long session of training and comparing performances (the less the better) for different functions, I have finally created five neural networks with the following sets of MATLAB functions:
1 newcf trainlm initnw mse learngd satlin
2 newcf trainlm initnw msne learngdm compet
3 newelm trainlm initwb mse learnhd purelin
4 newff trainlm initnw mse learngd purelin
5 newff trainlm initnw mse learnwh tansig
The best performance results (perf) for each one of them vary from xxxE-30 to xxxE-32.
But still, after running simulation of those networks for each single column of the input matrix, I got the expected output results in just 60% of the cases, while the other 40% are totally wrong.
I have exactly the same 60%/40% relationship between good and bad simulation results for all the above networks, with different bad columns per net.
Can something like this happen? What do you think could be wrong? Maybe the perf result when training is not enough to judge when a neural network is good enough? Maybe I didn't understand something well in the concepts?
Thank you in advance.
[edit 26/04/2013] The approximate code I am using :
inp = {
0,1300 0,0300 0,0300 0,0300 -0,0100 0,0300 0,0900 0,0100 0,0600 0,0700;
0,0500 -0,0400 -0,0400 -0,0400 -0,0100 0,0100 0,1400 0,0900 0,0600 -0,1700;
0,2400 0,2200 0,2200 0,2200 0,3700 0,3000 0,5300 0,3400 0,1400 -0,6700;
.......
}
outp={
0,0427 -0,1071 0,0605 -0,0637 -0,0410 0,2566 -0,0551 -0,0902 -0,2483 0,1543;
-0,0249 0,0192 -0,1199 -0,3748 0,3212 0,5490 -0,1655 -0,1213 -1,0236 0,4678;
.......
}
[out_r, out_c] = size(cell2mat(outp));
[inp_r, inp_c] = size(cell2mat(inp));
%------------ for 2 layers:
biasConnect = [1;1];
inputConnect = [1; 0];
layerConnect = [0 0; 1 0];
outputConnect = [0 1];
%-------------or for 3 layers:
% biasConnect = [1;1;1];
% inputConnect = [1; 0; 0];
% layerConnect = [0 0 0; 1 0 0; 0 1 0];
% outputConnect = [0 0 1];
my_net = network(1, 2, biasConnect,inputConnect,layerConnect,outputConnect); %create neural network
my_net.inputs{1}.size = inp_r; % set the number of elements in an input vector
my_net.outputs{1}.size = out_r; % set the number of elements in an output vector
my_net = newff(inp, outp, [inp_r out_r]); % Create feed-forward backpropagation network
% or for 3 layers: my_net = newff(inp, outp, [inp_r round(inp_r/2) out_r]);
my_net.layers{1}.size = inp_r; % 1st layer size
my_net.layers{1}.transferFcn = 'purelin'; % transfer function
my_net.layers{1}.initFcn = 'initnw';
% ---- Functions ----------------------------------%/
my_net.divideFcn = 'divideblock';
my_net.plotFcns = {'plotperform','plottrainstate'};
my_net.initFcn ='initlay'; % Layer init function
my_net.performFcn = 'mse'; % performing function
my_net.trainFcn = 'trainlm'; % training function
my_net.adaptFcn = 'learngd'; % should be from list: learngdm, learngd
% ---- set a few traininig params and train the net
my_net.trainParam.epochs = 100;
my_net.trainParam.goal = 1.0000e-030;
my_net.trainParam.max_fail = 3;
my_net.trainParam.mu = 1.0000e-03;
my_net.trainParam.mu_inc = 10;
my_net.trainParam.mu_dec = 0.1000;
my_net.trainParam.mu_max = 1e10;
my_net.trainParam.showWindow = false;
my_net.trainParam.showCommandLine = false;
my_net.trainParam.show = 0;
[my_net,tr] = train(my_net,inp,outp); % train the network
% ---- After all when the best my_net is found, I perform simulation for each column:
Y = sim(my_net, inp{:,i}); % for each i-column of the inp matrix and expect y=outp{:,i}