I'm training a neural network for a particular problem which can be predicted with 100% accuracy. However, the problem is that results tend to vary between 99% and 100% even if I train 10 different networks and take the average. I was reading around and found that it was due to random weights initialized every time the program ran. Is it okay to set the random seed to 1 and then train 10 different networks and take the average? This will result in less variation and easy model selection.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
You're trying to hide the problem under the rug. The variability is not a simple annoyance. If your model selection depends on the seed, then fixing the seed only hides the issue. Your models are not different enough for the sample you have, that's the bottom line.
-
$\begingroup$ I see your point. I read it somewhere and thought it was a viable solution. However, averaging did solve the problem on my previous neural net. I'll keep working on the model to get more consistent results. Thanks! $\endgroup$ Commented Oct 14, 2015 at 18:00
-
$\begingroup$ @HassanAbdulQayyum - if you want the neural network to act well on unknown data (generalize) then you need to pay attention here. Picking a particular "set seed" is like weighing the dice - they are no longer random and so they will not do their job. Look at making a hold-out set for this approach - train the 10 networks on 80% of the data, and then test on the held-out 20%. This will tell you if you are 100% (doubtful) or if you are "hiding the problem under the rug". $\endgroup$ Commented Oct 14, 2015 at 19:40
-
$\begingroup$ @EngrStudent the training data is 891 examples and testing data is 418 examples. $\endgroup$ Commented Oct 14, 2015 at 19:52