I have had some success training my deep neural network (with ReLU hidden units) by first normalizing the features of my data set to zero-mean-unit-variance.
Each sample of my data set has 600+ values. Majority of them are continuous but a handful or columns are symbolic / discrete (represent by fixed range integers). Currently I just normalize all columns to zero-mean-unit-variance the same way. Should I only normalize the continuous value columns? Are there better data normalization strategies?