# Using cosine of measurement time as feature for decision trees VS NNs

I have a regression data set and I'm trying to do some feature engineering.

The data set is foot fall coming into a store measured on the hour.

I'd like to include the time of measurement as a feature and I think that mapping the time of day onto a cosine curve should do this.

The target (in_coming) peaks at around 2PM. However my cosine curve peaks at 12PM. I think that this isn't a problem for neural networks as the bias values should easily be able to introduce a phase shift to the time feature aligning it with the peak of the in_coming. I think that if I manually tried to move this peak during feature engineering it would be passing information about the target values behaviour into the training data. So i will leave the cosine peak where it is.

However, I'm not sure if a decision tree can compensate for this offset in a similar way? It's likely that I will end up investigating the use of XGBoosted regression decision trees so would like to know if the time-cosine feature makes sense with trees or whether I should use some other method for time.

I've been investigating this myself, yes a decision tree is quite capable of learning a shifted cosine term, i.e. using catboost and python

t = np.arange(0,24,1) w = 2*np.pi / 24 p = -8*w
X = pd.DataFrame({"S1" : np.sin(w*t),"C1" : np.cos(w*t),})
y = np.sin(w*t + p)

from catboost import CatBoostRegressor
estimator = CatBoostRegressor(logging_level='Silent')
estimator.fit(X,y)
plt.plot(estimator.predict(X))


results in a reasonable approximation to the original y.

You can also just regress y ~ t, the fit isn't nearly as good though

X = pd.DataFrame({"t" : t})
y = np.sin(w*t + p)

from catboost import CatBoostRegressor
estimator = CatBoostRegressor(logging_level='Silent')
estimator.fit(X,y)
plt.plot(estimator.predict(X))


It's not entirely clear to me why the former results in a smooth approximation and the latter isn't. In general trees generate a piecewise constant approximation. Boosting smooths the apporixmation but it looks like catboost only grows each tree to depth 3 in the 2nd case and up to 5 in the first case. Also in case 2 after about steps the fit is as good as it gets, in case 1 it keeps improving up to ~1000 steps (I set learning rate to 1, and played with n_estimators and max_depth)