There are lots of discussions and research about tips and tricks that are helpful for convolutional networks training. (Like in this paper). The new architectures/optimizers/layers emerge very often, especially in the fields of computer vision and NLP. I wonder is there something similar for the fully connected networks applied to the tabular/structured data classification?

The most widely used approach I see is to use embeddings layers, concatenate them with numerical input, and pass it through the stack of fully-connected layers (probably with dropouts and batch normalization). Are there any other recommendations or state-of-the-art methods applied for the tabular data, any custom architectures suitable for this task? Or the "simple" approach outlined above is the best available solution?

I understand that people mostly use boosted trees or linear models in this case but I would like to investigate possible solutions available in the Deep Learning world.

  • 1
    $\begingroup$ While the focus is on debugging networks, there are a number of tricks suggested here: stats.stackexchange.com/questions/352036/… $\endgroup$ – Sycorax Jan 27 at 16:33
  • $\begingroup$ Very thorough and elaborated post! Thank you. I'll definitely go through it. $\endgroup$ – devforfu Jan 29 at 15:13

Yes, there are lots of them. I haven't read the second edition (published 2012), but the first is very good.

enter image description here

Somewhat old now, but the Comp.ai.neural-nets FAQ list also has plenty of good advice that is still valid today.

| cite | improve this answer | |
  • $\begingroup$ Great, thank you! Actually, some time ago I've already encountered the reference to this book somewhere but didn't pay proper attention to it :) $\endgroup$ – devforfu Jan 29 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.