Correct way of mean-centering and scaling time series data used as input to an LSTM?

I am training an LSTM network in tensorflow/Keras which takes eight different time series/features as input. The input matrix given to the network has the form ( nSamples x nTimesteps x nFeatures ).

My question is: when mean-centering and scaling the data to unit variance before giving it to the network, should one mean value and one standard deviation be calculated for every feature of the dataset, or, should a mean value be calculated for every individual timestep of each feature?

Here is a Numpy example of the alternatives I am contemplating: say my data to be preprocessed is stored in a three-dimensional matrix X.

One way of mean-center and scaling X would be:

mx = np.mean( X, axis = (0, 1) )
stdx = np.std( X, axis = (0, 1) )
X_scaled = ( X - mx ) / stdx


But another way would be:

mx = np.mean( X, axis = 0 )
stdx = np.std( X, axis = 0 )
X_scaled = ( X - mx ) / stdx


In the first method of the example mx and stdx are of shape ( nFeatures ). In the second method mx and stdx are of shape ( nTimesteps x nFeatures ).

If all columns of X (all timesteps) represented lagged values of one data source (data from one sensor for example) they would be expected to have the same mean and variance, and thus the first alternative in the example makes more sense which is also recommended here.

However, the nature of my input data is such that the timeseries consists of sorted data that is always increasing. For example the content of X[0,:,0] could be: [10, 20, 100, 500, 2500]. Which means that the columns of the X matrix do not share a common mean or standard deviation; the mean value is increasing for each timestep. Does that mean that alternative two, which considers the data of each column separately, is the appropriate way of scaling this type of data? Or is alternative one still the correct way to preprocess the data?

Sorry, it seems that I am about 2 years late to this post, However I figured I would give my opinion now in case that someone runs into the same question later (The same way I did today). The short answer is: option 1 is best, because you risk losing information with option 2.

Now the long answer:

The first thing that must be noted is that you are doing a mean-removal type of normalization (By subtracting the mean out of your sample). This would likely break any time-dependent trends that you may have, because now the data at any timestep is centered around zero (without regards to any other timestep).

For example, let's assume that at your first timestep the mean sensor reading of all your samples is 10, then at the second it is 20, and so on following your example array. When we normalize across each time step then your example array will now be just a zero vector. if instead we normalize across the entire feature, then your new array will preserves the rising trend in the data.

In the response to the StackExchange that you linked, the responder explains that you should use the first option in general cases; However, in cases where you may want to use option 2, you need to make sure that you have another feature that contains information about your scale (mean and variance) at that timestep.

This is where your human judgement must come to the rescue. What information do you think is valuable to the model? If the rising trend of the time series data is not important then perhaps you should not be using a sequence model at all. If the variance of the data at each timestep is significantly different, then perhaps you should make this its own separate feature (as seems to be the case for the Heston model in the linked StackExchange). Feature engineering is often a very case specific problem. Therefore you need to try to understand your data very well, and understand the different factors that affect its trend.

Finally, if you are still unsure, it doesn't hurt to train a model for each option for 25 epochs or so. That should be enough training to see significant differences between each option.