I have a fair idea that a lot of research has been done and is still underway to explore the science behind the black art of a neural network (NN) architecture, i.e., accurately calculating the number of hidden layers and the number of neurons in each layer in an NN and we are yet to find a definite answer. I as part of my research developed a problem specific feedforward NN for binary classification (let's name it NN-1). The input data comprises of the 'Received Signal Strength' and/or 'Time of Flight' of the users transmitted signals recorded at static base stations. Now I am trying to defend the architecture of NN-1, i.e., that no other new NN architecture (which if designed with a different number of hidden layers and/or changed number of neurons in the hidden layer as compared to NN-1) can beat the performance (on test set) of NN-1. *Do note that training and the test sets for NN-1 and other NNs (with changed architecture) are the same. *

Few ideas I am considering are:

  1. To formulate a set of mathematical equations that helps in calculating/recommending an optimum number of hidden layers and the number of neurons in each layer for NN-1. It would work even if the derived mathematical equations are only valid for my specific classification problem. Once successful, I can modify NN-1's architecture accordingly and can confidently state NN-1 to have an optimum architecture. I would welcome if anyone can guide here or can point out to any related work (which has explored such mathematical approach towards determining the number of hidden layers/neurons in an NN) so that I can further capitalize on that. Please note that I have read a lot of literature during the past few months but have only come across guidelines on how to choose the number of hidden layers and the number of neurons in those layers.
  2. Independent of the above, I look forward to any ideas following which can help me in defending the NN-1's architecture.

Thank you

  • 1
    $\begingroup$ I am a little bit lost with your question. On one hand, you want to write a paper on a very complicated subject, that as for now, lacks this kind of definitive answers. On another, you seem to be asking us to provide you with the very basic building blocks to write it. Maybe tell us where are you at this moment? What literature have you already checked and what are your findings right now? What kind of help do you need? $\endgroup$ – Tim May 17 '19 at 6:27
  • 1
    $\begingroup$ Moreover, what does "no other NN architecture can beat the performance of the designed NN" exactly mean? Beat on training set? Or on held-out test set? If the second case, I bet that someone could prepare such training and test set that would lead to your network performing worse then something else. $\endgroup$ – Tim May 17 '19 at 6:36
  • $\begingroup$ This question is a bit misguided. What you can do to defend is say due to the size of certain features we chose these filter sizes or this feature pyramid network. Any mathematical expression of such would require to estimate the complexity of the data per the task. And since we cannot capture that, why NNs exist, trivially what you are saying is not possible. Even the estimator would have to be a NN! $\endgroup$ – Rahul Deora May 19 '19 at 12:14
  • $\begingroup$ Its like saying prove polynomial regression with degree K is better than any other degree. $\endgroup$ – Rahul Deora May 19 '19 at 14:19
  • $\begingroup$ I greatly appreciate all the inputs. I have tried modifying my question to avoid any confusions. @Rahul Deora: I agree to your comment but can we think on the lines where the performance evaluations for the NNs (NN-1 and any other NN designed with a different architecture) is based on the SAME test set. Moreover, all the NNs are also trained using the same training data. $\endgroup$ – Skyward May 20 '19 at 5:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.