Imagine you have many observations on which you want to run a classification algorithm. Each observation is characterized by a matrix of non-negative values. For all observations 90-98% of the values are 0.
To ensure that a machine learning algorithm perform the best, it is often recommended to do a feature standardization (see e.g. http://ufldl.stanford.edu/wiki/index.php/Data_Preprocessing). However, with a normal feature standardization the sparse cells obtain a value of roughly -0.25. I'm really interested in the benefit (both learning-wise and computationally) by letting all the cells with 0 stay 0 so the matrix continues to be sparse. One scaling method that achieves this is the 0-1 scaling. However, that is problematic if there's some extreme outliers in the data, which there are.
The data is modeled using a convolutional neural network, which I'm training with a stochastic gradient descent algorithm. What kind of feature scaling would make the most sense from an empirical and theoretical point of view?