# Average value prediction for Artificial Neural Network

In some research paper, there are researcher used taking ANN prediction by run it multiple time and find the average result for prediction. Is it necessary to make it that way?

• What exactly do you mean by "run it multiple times"? Do you mean to train the network multiple times? – Hugh Jan 15 '17 at 10:31
• @Hugh I train the network multiple times and the result i obatin i take the everage – bbadyalina Jan 15 '17 at 13:15

Taking the average of the predictions from several neural networks trained separately is a common ensemble method. It often slightly improves the prediction, e.g. see Ensemble of convolutional neural networks for pattern recognition tasks?.

Some visual explanation from http://images.slideplayer.com/17/5270015/slides/slide_2.jpg:

(unlike what the slide says, the training data and the learning algorithms do not have to be different.)

It is a interesting question, I think the way to take the average result for the prediction is necessary.

run it multiple time and find the average result for prediction

It seems that this is seemingly like cross validation or make the result less randomness.

The key thing to keep in mind about any neural net model (including all the deep learning varieties) is that the solution you get out of gradient descent is merely a local minimum. The loss function is non-convex.

By running it multiple times and taking the average (instead of the best) you hope to get something better than a single solution.

Some researchers in deep learning (e.g. Yann LeCun) have noted that this local minimum issue is probably not so problematic; they observed that typically a local minimum found in a single descent trajectory turns out to be pretty good.