I have 3 integer valued time series $a_t$, $b_t$ and $y_t$ with $k$ observations. I want to fit $y_t$ with the 2 first, and for that purpose I use a regression tree like this:
- test all combinations of $a_t,\ldots a_{t-k}$, $b_t,\ldots b_{t-k}$, for each value between 0 and $N$. For each combination, take $\bar{y}_s$ the mean value on the subset as an estimator,
- select the splitting variable/value that minimize $\sum_{i\in s}(y_i - \bar{y}_s)^2$
I stop the recursion when the tree is deep enough or if I don't have enough point to compute $\bar{y}_s$.
The distribution being very peaky and the dataset big, some combination of $a_t,b_t,a_{t-1},b_{t-1},\ldots$ are extremely frequent and some of them appear just a few times.
While $\bar{y}_s$ seems to be a good estimator on the training set, it degrades very quickly on the tail of the distribution on the test set (but it is good on the test set for the center of the distribution).
How to improve my estimator in my regression tree to better fit the tail of the distribution and keep a constant performance on the tail of the test set too ?
