I have a time series dataset including both new and repeat customer interactions. I noticed buyer behavior is dramatically different between the two segments, with repeat customers highly depending on past interactions while new customers depending on other factors.
I would like to create a stacking ensemble that creates multiple base estimators, some related to repeat and some to new customers.
However, there are very few initial repeat customers and their count starts to become significant only half way through the series. Is this a problem potentially? Would I need to have my top layer classifier in the ensemble automatically learn to disregard the repeat customer base estimators in the first phases of the time series? Should I have the repeat customer base classifiers output some nulls when there are no repeat customers?