Suppose I have two ANN architectures, the first one has five neurons in the hidden layer while the other has 10 neurons. My data is shuffled and divided into training, validation and testing sets. I came across a paper where the authors have done the followings
1) Using Levenberg-Marquard algorithm; Section 4.1 page 6 of Application of soft computing methods for predicting the elastic modulus of recycled aggregate concrete. Therein, to decide the best architecture, run each ANN architecture 20 times with the SAME training data (The data is not shuffled in each run) but random weights at each run, then select the architecture that gives lowest RMSE.
Q1 What is the problem with this approach compared to having the data shuffled in each run?
2) Using an extreme learning machine (ELM); section 4. page 6 of Predicting compressive strength of lightweight foamed concrete using extreme learning machine model Suppose the number of hidden neurons is decided. To compare the accuracy of ANN with other methods. The authors, because of the random selection of the initial weights of ANN, ANN runs 20 times (The data is not shuffled in each run, Only the weights are initialized randomly in each run), and they report the results of best run (not the average, even though the data is shuffled once)(best run means, suppose at run 18, ANN gives the best result).
Q2 Is this approach is valid? If not, Why? If they have reported the average of the results, is this approach, of using the same shuffled data in multiple runs, reliable compared to shuffling the data in each run?