I am working on employee attrition prediction with raw data that has more than 1 year's amounts of daily metrics, and have some confusion on the best data aggregation techniques/methods for a successful prediction/forecast model. (I have gone through several other related posts, including Survival Model for Predicting Churn - Time-varying predictors? and Selecting the best time lagged moving average for time series analysis, but haven't found enough information for my questions and scenario here)
I am currently using a lagged time period of 1 month along with weekly averaged metrics for the forecast, so for example, I am using the weekly aggregated metrics at week 1 to predict employee attrition at week 5. In my data aggregation, for employees at week 1 I would find if they have attrited at week 5, and if so I give them a positive attrition label along with their week 1's weekly aggregated metrics and, if they have not attrited then a negative attrition label. However, I am using over 1 year of raw data to create this aggregated dataset, and there are a couple of concerns I have:
- For each set of attrition labels I get over 4 weeks period, the employees may not be the same, and this can result in distribution shift (or temporal bias). For example, if we get attrition labels from week 5 for employee 1 and employee 2 using their aggregated week 1 metrics, and employee 1 leaves the company at week 6, then when we get attrition labels from week 11 using aggregated week 7 metrics we would only have data from employee 2 (since employee has attrited and doesn't have any daily metrics anymore after week 6). Would this be a big concern?
- Since I am using a very large time window (>1 year), there is the possibility of distribution shift. For example, we have two data that have drastically different weekly averaged metrics due to holiday or other reasons, and both of them have positive attrition label, then these data together may not be indicative of the attrition (I am just giving examples but in reality if a lot of such situations arise then that could be problematic)
In short, I am wondering if anyone sees any problems with my approach here for my attrition forecasting, and if possible I would like to get an idea of the optimal lagged time and other associated parameters for this modeling. Thanks!