3
$\begingroup$

I want to list the advantages and disadvantages of Neural network methods for classification or estimation purposes. I have already found the advantages of NN method in many papers. But they didn't write anything special about disadvantages of NN method as it was their own proposed method ;) . I guess they have some disadvantages too. For example as we know the performance of NN method is based on some settings, like number of hidden layers nodes for MLP neural network, Also there is no analytical method to fine-tune these settings except trial and error. Am I right? If yes, Do you guys have some references about this disadvantage? Please give some strong references(Books or papers).

Please also do tell if there are other disadvantages exist and provide some references for them.

$\endgroup$
3
  • 1
    $\begingroup$ You might want to read arxiv.org/abs/1312.6199 - intriguing properties of neural networks. ..showing how minor changes to image can cause significant misclassification $\endgroup$
    – seanv507
    Sep 21, 2014 at 22:23
  • $\begingroup$ I read it . But didn't find what wanted. @seanv507 $\endgroup$
    – SAH
    Sep 22, 2014 at 6:11
  • $\begingroup$ So you have to read it understanding that its by neural network researchers. Its showing that the black box feature of nns means that it is quite unpredictable what will be classified correctly-changes that are invisible to the human eye lead to incorrect classification. This is the problem with the black box approach.if you dont know why it works, you don't know when it will break down. $\endgroup$
    – seanv507
    Sep 22, 2014 at 6:22

1 Answer 1

2
$\begingroup$

Neural nets have many advantages, but here are some disadvantages:

Large number of hyperparameters. This includes network architecture (how many layers, layer size, layer type), activation function for each layer, optimization algorithm, regularization methods, initialization method, and many associated hyperparameters for each of these choices. Hyperparameters can strongly interact with each other to affect performance. Good settings are highly problem dependent, and there typically isn't a clear, obvious choice. Hyperparameters are typically chosen using some combination of intuition/experience, copying past work, manual tweaking, and black box hyperparameter optimization algorithms. Training large neural nets on large problems takes a long time, so tuning the hyperparameters can be very laborious and/or computationally expensive.

Non-convex loss functions. Optimization for neural nets is trickier than for methods with convex loss functions. We don't get the nice guarantees that come with convex optimization. Additionally, we have to deal with issues like local minima and saddle points, which can trap the optimization algorithm at bad solutions. Optimization for neural nets is an active research topic, and much work has gone into trying to mitigate these issues.

Black box model. Neural nets implement complicated, nonlinear functions that are not straightforward to interpret. This is fine if we only care about the output (e.g. prediction in classification and regression problems). It's a disadvantage if we want to understand something about the mapping between input and output. Some methods for peering into the black box do exist.

Little theoretical understanding. Neural nets are less well understood than other methods, from a theory standpoint. Naturally, this isn't an inherent disadvantage of neural nets, but of our current understanding. Hopefully, this will improve with time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.