Is feature transformation (power, log, Box-Cox) necessary in deep learning?

Question

I've read that it's beneficial to apply certain common feature transformations on datasets before they hit machine learning models. These based on the distributions of the dataset's features; eg, applying log transforms to skewed normally-distributed features. Some examples here.

Now as I understand, a main boon of deep learning is "automatic feature engineering" (aka, "feature learning"). I know that includes feature combinations; but my hunch says that also includes learned feature transformations per the above? So when using deep networks with well-tuned hypers, can feature-transformations safely be removed from the human's responsibilities - that is, throw all this log/square/box-cox stuff away?

[Edit] Extra: does this also handle "feature selection" (deciding which inputs not to include) for you?

Answer 1

The rule of thumb is: the more data you have available, the less you have to care about feature engineering (which is basically inputting some prior knowledge into the model, based on the domain expertise).

Theoritically (with huge enough number of the samples) you could solve imagenet without using any convolutions, only deep feedforward network. But by knowing that pixels are spatially correlated (which makes it so that the convolutions will be much better way to tackle this problem) you can design an algorithm which is much more data-efficient.

Answer 2

So the way I view feature engineering ala-box cox is that we have a model that requires normality, we don't have normal data, so we do a transform to get us to normal Data. So on the one hand its true that neural network do not require normalized data so why feature engineer. On the other hand, while a neural net might eventually get there, sometimes feature engineering done by humans can hugely help the initial convergence rate. For example, in case of multichannel signal data, doing the Fourier decomp and computing the cross correlations beforehand greatly increases the speed at which the Neural Net can get to classification (to give a really specific example). Or to give a more sane example, if you know your data has many outliers and these are not important, removing outliers is a form of feature engineering. The network could eventually learn to ignore then, but it might take forever. So when you are fairly sure that the transformation is going to highlight something important about your data, then transform it, if not, then maybe not.