I'm want to build a numerical regression to forecast.

From my initial analysis, it shows linear models (glm) out performs the typical decision tree models (xgboost, ranger...etc). I hypothesized that 2 factors may be at play. Features (independent variables) truly follows a Gaussian Distribution (bell curve) and thus will outperform non-linear models and that decision trees do not extrapolate as well (expecting incremental volumes not seen in historical data).

Due to the complexity of the number of features and it's high levels of interaction effects, I'm keen to try Tensorflow. As i understand it, it can extrapolate well, as it uses many coefficients as per the linear models. I'm experiencing issues with the pre-processing of data. Literature suggest that scaling is important, and my understanding is two types of scaling is generally used. Min Max Normalization and Mean Standard Deviation (Standardization into Z-scores) (see https://machinelearningmastery.com/how-to-improve-neural-network-stability-and-modeling-performance-with-data-scaling/).

As I want to introduce more features into the model (including non-linear ones), I'm going to use the min max normalization. The issue is, due to the continued incremental numerical values (both independent and dependent variables), It doesn't seem right to normalize new datasets with historical min max values.

How should I pre-process (normalize / scale) when i expect future datasets to have increasing numerical values?

Simple example of the scenario described above would be.

Train dataset (2 columns x 10 rows, 1 target, 1 feature)
target values ranges from 1 to 10
feature values ranges from 30 to 50

validate dataset (2 columns x 10 rows, 1 target, 1 feature)
target values ranges from 11 to 20
feature values ranges from 50 to 70

Notice that the validate data has an increasing target, and feature range.

The question now is, if normalization / scaling is required as preprocessing prior fitting a tensorflow model, what impact would an increasing numerical values for both target and feature has on model performance based on validate datasets?

hope the above make sense.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.