I had run artificial neural network on Matlab. Although i used the same design structure of ANN and the same data set, the result always different. Some suggested using ensemble neural network. From my reading ensemble is combine ANN with different design structure. Do this applicable in my problem. Other than that, why is ANN produce different result every time i run it?
Why ANN produces different results: this is probably the training procedure involves "randomness", for example,
- Your training may use random parameter initialisation.
- Your training may use random dropout as regularisation.
- Your training may use SGD and shuffle the data order every epoch.
And ANN is a not a convex function, so any of these "randomness" may lead to different local optimum.
And ensemble method is a very general method to reduce the variance of predictor then improve the performance. It is not necessarily specific for ANN. And there are many ways to train an ensemble of ANN (like using different datasets, using different initial parameters, using different structure/dropout). This is really an "art" which means you need to try a lot and pick the best way for your specific dataset.
Your second question is more simple so I'll answer that first.
When you train an ANN you start with random weights in the network. The network is trained to try to minimize the cost function of training. We usually cannot find the exact minimum of the cost function during training so each time you train an ANN it starts with different weights and ends training with different weights.
Training a single ANN is quite costly so it could be difficult to train 10 ANNs and make an ensemble of them all. Instead we use a clever trick called dropout when training one ANN, dropout is a type of regularization which acts like training an ensemble of networks inside one network.
Here is a description of dropout, scroll down to read about dropout.