I'm doing a study on one dataset that contains 70 financial ratios for all U.S. companies across eight different categories (Valuation, Liquidity, Profitability, and etc) from 1970 to 2018 monthly. Index of the data has two levels: year-month and stock ticker. My goal is to use these features to build a multiple regression model on stock returns.

One of the challenge in data cleaning is handling NaN values. Here is what I did. To avoid look ahead bias, I used a forward fill. However, this will not fill all the NaNs in the dataset. Some early observations have no prior values to forward fill.

So after forward fill I standardize each feature by computing the Z score. Finally, fill the rest of the missing value with 0.

If I do this, I effectively filled NaNs with the mean, which is 0 after normalization. Is my foward_fill->Z_Score->fill_zero ordering correct? Are there any pitfalls or will this incur bias? What is the best practice in my particular case? Any reference will be greatly appreciated.

  • $\begingroup$ Any insight on handling time series without look ahead bias will be appreciated $\endgroup$ – Kemeng Zhang Apr 10 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.