Why not to normalize the image to have zero mean in reinforcement learning? I learned in deep learning classes, cs231n for example, that it is better to have input data to have negative values so as to have negative gradients. Although negative gradients may also be achieved deep neural net, this, to my best knowledge, is still recommended in compute vision tasks. But I've recently seen many reinforcement learning algorithms on Atari, such as OpenAI's baselines, that do not zero center the input data --- they simply divide the input images by 255. What makes this difference? Why don't they zero center the data? Which is is better in practice?
 A: In reinforcement learning (RL) it is not common to normalise inputs to mean $0$ standard deviation $1$, because the problem is non-stationary, plus data for training is generated as required. Due to these traits, you don't have access to a large unbiased data set where you can establish accurate bounds on statistics for the input features and outputs, except towards the end of training close to convergence.
For this reason, you won't generally see use of standard feature normalisers for deep RL. However, neural networks are still sensitive to scale of input features, so at least some basic scaling needs to be done.
That does not stop you for example dividing by 128 and subtracting 127 for normalising pixel values. This should work fine for training ML based on neural networks, and make the value 0 refer to a mid grey colour. Other scalings could also work, and you could based them loosely on general statistics of the images found in the target environment.
Scaling of pixel channels into the range $[0,1]$ is often convenient for other purposes, such as displaying the images from some code libraries. It is also a convention that has been used in some original RL papers, so it may be continued to be used for fair comparison between techniques.
