I have a
(X,Y)=(100,5) dataset (non-image) that I used with a deep linear classifier on
Tensorflow to train and evaluate. At the same time, I have tested the very same dataset with conventional and shallow ML models (i.e. Random forest, DS-trees, MLP, etc.). I am seeing no tangible improvement in accuracy, and in some cases, even a decrease on accuracy level using my deep model.
Now, I was wondering whether there are lower-bounds involved with using deep models. I know there are several (CNN, RNN, etc.) models and generalization is very coarse-grain, however, I would like to have a rule of thumb when we deal with low dimensional data as
features (Y-columns) and