I'm using some machine learning models to predict future values of a time series (stock returns). In the data preprocessing step I'm standardizing all variables (incl. the target variable) using df_std[features]=sklearn.preprocessing.scale(df_std[features], axis=0, copy=False). I'm however not sure if i am inducing look ahead bias since i'm using the mean and standard deviation of the whole dataset. However, i do this in the data preprocessing step before dividing the data in training and test data and before introducing the models. I'm realy tring to wrap my mind around it.

  • $\begingroup$ Hi: That would depend on the horizon you are using for prediction. are you predicting say the next week's return, the next day's return, etc. Essentially, as long as the data you use to make a prediction does not overlap with the actual return that you're predicting, then you're okay. If they overlap, then you've got a bias. $\endgroup$
    – mlofton
    Nov 14 '21 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.