I am attempting to use a neural network, after using other machine learning algorithms. I am using the RSNNS package (I am willing to use / evaluate other packages) that's part of R. I would like to get a precision that's at least 66%.

I split the data in a training and test sets, with 4/5 of the data in the training. I then trained models using different network layouts and learning rates. I selected the parameters that gave precision >66% and the largest F-measure.

The parameters I selected gave a precision of 70%. I then took the data and did a 10-fold cross validation using the same network layout and learning rate. With this k-fold cross validation, I get a precision that is just above 50% (which is similar to the other algorithms I used).

My question is, is the 70% precision accurate with the test set? Is my k-fold validation possibly finding local optima, and not giving an accurate precision?