I am building a boosted decision trees classification model, where the input variables vary smoothly with time.
The problem is that the predictions will always be biased by the most recent entries. I understand that this is happening because these are the points of hyperspace closest to the target points. If I don't use the n last entries, then the predictions are still biased by the last used entries.
How can I avoid this bias and "shake things up"? I expect that the correct model will pick up relations between variables and use them, not blindly looking for the nearest hyperpoints.
The solution could be to use another ML method, but some quick tries show that the problem isn't easy to get rid of, in general. So, any suggestions, either within BDT or with another algorithm, are appreciated.