1
$\begingroup$

I have a (X,Y)=(100,5) dataset (non-image) that I used with a deep linear classifier on Tensorflow to train and evaluate. At the same time, I have tested the very same dataset with conventional and shallow ML models (i.e. Random forest, DS-trees, MLP, etc.). I am seeing no tangible improvement in accuracy, and in some cases, even a decrease on accuracy level using my deep model.

Now, I was wondering whether there are lower-bounds involved with using deep models. I know there are several (CNN, RNN, etc.) models and generalization is very coarse-grain, however, I would like to have a rule of thumb when we deal with low dimensional data as features (Y-columns) and instances (X-rows).

$\endgroup$
  • 3
    $\begingroup$ Keep in mind that neural networks of any kind will overtrain like crazy. So play around with your layers, either by increasing regularization or by using fewer layers with fewer connections. $\endgroup$ – Alex R. Mar 17 '17 at 17:15
1
$\begingroup$

In theory, the more variables in your model (the more complexity) means that you will need more data to train it effectively. If you do not have a lot of data than I suggest to stay clear from any deep technique. Don't just use them because they are 'hot' right now. Use deep learning models only when they are appropriate to the problem you are trying to solve.

As a general rule of thumb I tend to follow the following for deep models:

Data points needed = (# of features) * (# of classes) * 100

For shallow models

Data points needed = (# of features) * (# of classes) * 10

This usually gives pretty reasonable results. However, in many cases, if you are trying to learn a very complex function, then you will need many more examples when you are training your deep model.

I would suggest you stick to shallow machine learning techniques. Try SVM, it is one of the most powerful techniques and often fares very well.

$\endgroup$
1
$\begingroup$

Your classes are probably linearly separable, since use of more advanced classifiers does not help -- the shallow classifiers can discern the class separability. There may be few non-linear associations as well, however, if there are, the shallow classifiers can discern the patterns.

Would not recommend starting with complex methods first, since you are fighting Occam's razor and statistical parsimony: "don't use anything complex if simple methods work."

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.