Artificial neural network analysis

Question

I've designed and programmed an ANN, So now I need to do an analysis about results, I've seen some examples where they do graphics about surface error, average of true outs, and another interesting graphics so I've decided do it, I think to use Gnuplot to do this analysis, but I need some advices or things that i do with the data before plot these. I don't know if I could plot the data as I got or I've to preprocessing before. I've seen this page where are some scripts of data representation (example).

More specifically I need to know what kind of analysis I have to do for get information about the efficiency of my network.

Answer 1

As prelic points out, whatever error metric you use, make sure you measure it out-of-sample. You should do this through cross-validation, but you can also use a held-out test set if time does not allow for cross-validation.

Your error metric depends on what sort of data you are dealing with. In general, there are 2 kinds of problems in machine learning: classification problems and regression problems (though this is not always the case).

• For classification problems, it is useful to look at confusion matrixes, roc curves, and precision-recall curves.
• For regression problems, it is useful to look at RMSE or MAE, xy plots of your predicted y values vs the actual y values, and histograms of your residuals (predicted y-actual y).

Answer 2

Think about K-Fold Cross Validation, a common way to generate some analysis of your model, the results of which you can visualize however you want.