# Best practice for encoding datetime in machine learning

I'm working on a Kaggle problem (the problem closed some time ago, but doing it for self-study/practice) where the output is clearly affected by both the year and the month.

The original datetime data provides year/month/day/hour information and I felt that year and month were probably the only necessary data. So I've currently modified the feature such that the data is represented only by year and month ( ex) March of 2016 would be 201603) and graphed each outcome with respect to the modified time variable consisting of year/month pair.

As you can see here, the 1st outcome has some minor seasonal fluctuations whereas the 3rd and 4th outcomes have clear seasonal trends. On the other hand, the 2nd outcome drastically decreases after May of 2015 (201505).

For my model prediction, I'd like to somehow incorporate time as a variable in a way that makes sense. What would be the best approach here? Can I just assume the earliest time period in the data to equal to 1 and increment by 1 for every month and treat the variable as a nominal category variable? Or something else?

Thanks

• Check out Feature-Engine's CyclicalTransformer it performs Marks proposed implementation. Oct 27, 2021 at 16:43

You want to preserve the cyclical nature of your inputs. One approach is to cut the datetime variable into four variables: year, month, day, and hour. Then, decompose each of these (except for year) variables in two.

You create a sine and a cosine facet of each of these three variables (i.e., month, day, hour), which will retain the fact that hour 24 is closer to hour 0 than to hour 21, and that month 12 is closer to month 1 than to month 10.

A quick Google search got me a few links on how to do it:

• Thanks for the answer! Just one question though. I can see why this cycle transformation is needed for month information since we need the ML algorithm to recognize that December and January are indeed close to each other. But does the same logic apply for year as well? For something like month that cycles within a certain range of numbers, this approach makes sense, but year progresses with cycling.
– cck3
Nov 3, 2017 at 6:19
• You're right, my mistake. I updated my answer so that year is taken out from the list. The same logic does not apply to year as well. Nov 3, 2017 at 13:44
• I wonder how these can be applied when time/date are features - for example stackoverflow.com/questions/59653862/… Jan 16, 2020 at 22:06
• @user4581first thing that comes to mind is maybe put the data in long format, where each row would be a combination of observation/encoded datetime/the value of that datetime (a, b, c, NaN, etc.) Jan 16, 2020 at 22:15

I know I left a comment, but just to be explicit here is Feature-Engine's CylicalTransformer.

import pandas as pd
from sklearn.model_selection import train_test_split

from feature_engine.creation import CyclicalTransformer

df = pd.DataFrame({
'day': [6, 7, 5, 3, 1, 2, 4],
'months': [3, 7, 9, 12, 4, 6, 12],
})

cyclical = CyclicalTransformer(variables=None, drop_original=True)

X = cyclical.fit_transform(df)