Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have 3 integer valued time series $a_t$, $b_t$ and $y_t$ with $k$ observations. I want to fit $y_t$ with the 2 first, and for that purpose I use a regression tree like this:

  • test all combinations of $a_t,\ldots a_{t-k}$, $b_t,\ldots b_{t-k}$, for each value between 0 and $N$. For each combination, take $\bar{y}_s$ the mean value on the subset as an estimator,
  • select the splitting variable/value that minimize $\sum_{i\in s}(y_i - \bar{y}_s)^2$

I stop the recursion when the tree is deep enough or if I don't have enough point to compute $\bar{y}_s$.

The distribution being very peaky and the dataset big, some combination of $a_t,b_t,a_{t-1},b_{t-1},\ldots$ are extremely frequent and some of them appear just a few times.

While $\bar{y}_s$ seems to be a good estimator on the training set, it degrades very quickly on the tail of the distribution on the test set (but it is good on the test set for the center of the distribution).

How to improve my estimator in my regression tree to better fit the tail of the distribution and keep a constant performance on the tail of the test set too ?

share|improve this question

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.