I am trying to make "immediate" prediction of bikes availabilities (which means 30 minutes to 2 hours prediction from the last observation).
In my regression features, I found it clear that I had to put the "bikes_before" feature, that is the value of the last observation. To handle the lack of periodicity of the observation, I input another feature "timediff" that represents the time difference between the observation and the immediate last observation.
I expected that a good model would less take into account the "bikes_before" value when "timediff" is high, and in this case would take more into account the other features (weather, hour...). But I trained a gradient boosted trees model that gave me a ~1 bike RMSE (not surprising) and it gave a almost exclusive (~95%) feature importance to bike_before. When I artificially changed the timediff to a very high value for prediction, the model still predicts a value really close to the bikes_before value.
My question is : how can I make my model understand that "bikes_before" is a good information when timediff is low but not a good one when it is higher ?