Should you transform data (not just normalizing) before training a NN? Let's say I have stock price data. If I were to do linear regression, I would find transformations of the data that help performance. Should I perform comparable steps when training a neural network or is it redundant? I have generally have gone with my intuition that it will either improve performance or speed up training. 
For example, let us say I have stock prices, and earnings. If I were to perform some function between the two to create a third column of data that is some linear or non-linear combination of the two, assuming that this third column is meaningful, would this help the neural network?
Thanks for any enlightenment. Also, this is not about normalization of data. This is pre-normalization.
 A: People refer to this as "feature engineering".  It often helps.  Under certain conditions, neural nets can approximate any continuous function.  These conditions don't typically hold with modern nets (deep, rather than wide, and using ReLU-family activatons), but still neural nets can approximate quite a lot.  
But that doesn't guarantee that neural nets can do so efficiently.
Say the true model is 
$$
y = X\beta + \epsilon
$$
You don't observe $X$, but you observe $\mathbf{Z}$, where $X = f(\mathbf{Z})$, and $f$ is really complicated.  With infinite data, you can approximate $f$ pretty well using a neural net.  If you know $f$, or know it to some approximation, your neural net has to do less work in figuring out the optimal combination of $\mathbf{Z}$ to get $X$.  
In other words, a neural net (with a single layer for simplicity) is 
$$
y = a(\mathbf{Z}\Gamma)\beta + \epsilon
$$
Most of the work goes into finding $\Gamma$.  If you already know it, your process is more efficient, and you'll get better results.
