I have quarterly data of federal fund rate (test set), e.g.:

[2.18, 2.18, 2.18, 2.18, 2.18, 2.18, 2.18, 2.18, 2.18, 2.18, 2.18, 2.18, 2.19, 2.19, 2.19, 2.19, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.19, 2.2, 2.2, 2.2, 2.2, 2.19, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.2, 2.19, 2.2, 2.2, 2.19, 2.2, 2.19, 2.19, 2.19, 2.19, 2.2, 2.2, 2.2, 2.4, 2.4, 2.4]

Train data description is:

Mean: 1.003916
Variance: 0.203896
Std: 0.4514

I want to use this data as an input feature for the LSTM model. For this, I apply standardization, as follows:

I am forced to use training data mean and variance, since I need to avoid look ahead bias at dev and test sets. However, after applying it turns out that the data standardizes to even larger values.

This can't be used as an input to neural network, since it causes some features to be more weighted than others. Moreover, rest of features are within -1, 1 range.
I have also tried using Min-Max normalization, but in this case NN does not converge on some data points.

What should I do in this case?

  • 1
    $\begingroup$ The fact that NN doesnt converge on fed funds variable standardized with the range is a red herring. This cant be the reason $\endgroup$ – Aksakal Apr 23 at 20:21
  • $\begingroup$ standard scaling means dividing by the standard deviation $\sigma$, while your equation shows you dividing by the variance $\sigma^2$. After scaling you should check that $x'$ has mean zero and variance one; and you should expect to see a mix of positive and negative values roughly between -2 and 2. Also, the example data set you provide doesn't look like it has mean 1.0039 and variance 0.2039 like you claim. Unless the omitted values are very different than the ones shown, I would expect something like a mean of 2.2, variance .002, and standard deviation of .046. $\endgroup$ – olooney Apr 23 at 20:43
  • $\begingroup$ @olooney, you're right as stated above, I use mean and std from the train set and then apply to dev and test data(given above) $\endgroup$ – sokolov Apr 23 at 20:48

Usually you standardize by the volatility (std deviation) $\sigma$ not the variance $\sigma^2$. You want the unitless variable, but in your case the units stay. For instance, the interest rates are in inverse time units $year^{-1}$ so, your new variable is going to be in time units, years.

  • $\begingroup$ you're right, however I still get values within -3..+3 range, which are still not valid to be the input of the NN $\endgroup$ – sokolov Apr 23 at 20:14
  • $\begingroup$ I'm not sure why you think -3 and +3 range is not valid. If you really want to get them into $[-1,1]$ interval you either use the range instead of the standard deviation or as a quick fix get $3\sigma$ in denominator. $\endgroup$ – Aksakal Apr 23 at 20:16
  • $\begingroup$ I mean, that I have different variables on different scales, with different distortions, etc... Now, when I am using z-score, features may become to different ranges (e.g. from 1 to 4, or -3 to 3, etc.). If I use min-max scaling, then my NN is not converging (I have no idea why so). What I want, is to have input features more or less on the same range, so they have the same weight for the NN (without making some features more important than others). $\endgroup$ – sokolov Apr 23 at 20:22
  • $\begingroup$ It's likely that you have some other issues. Usually the exact way to normalize the data doesn't make a difference with non bounded variables such as interest rates. The notion of a scale is not as precise as you make it to be. When you standardize with a z-score you get variables in about the same scale. If this is throwing your model off, then something is wrong with your model. How are you going to predict with it if it's so sensitive to the slightest variations of a scale? Your inputs will not be perfect $\endgroup$ – Aksakal Apr 23 at 20:28
  • $\begingroup$ Maybe this might be the case... Can you take a look the model and tell me your expert opinion? I would really appreciate it. $\endgroup$ – sokolov Apr 23 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.