I'm working on a Kaggle problem (the problem closed some time ago, but doing it for self-study/practice) where the output is clearly affected by both the year and the month. enter image description here

The original datetime data provides year/month/day/hour information and I felt that year and month were probably the only necessary data. So I've currently modified the feature such that the data is represented only by year and month ( ex) March of 2016 would be 201603) and graphed each outcome with respect to the modified time variable consisting of year/month pair.

As you can see here, the 1st outcome has some minor seasonal fluctuations whereas the 3rd and 4th outcomes have clear seasonal trends. On the other hand, the 2nd outcome drastically decreases after May of 2015 (201505).

For my model prediction, I'd like to somehow incorporate time as a variable in a way that makes sense. What would be the best approach here? Can I just assume the earliest time period in the data to equal to 1 and increment by 1 for every month and treat the variable as a nominal category variable? Or something else?


2 Answers 2


You want to preserve the cyclical nature of your inputs. One approach is to cut the datetime variable into four variables: year, month, day, and hour. Then, decompose each of these (except for year) variables in two.

You create a sine and a cosine facet of each of these three variables (i.e., month, day, hour), which will retain the fact that hour 24 is closer to hour 0 than to hour 21, and that month 12 is closer to month 1 than to month 10.

A quick Google search got me a few links on how to do it:

  • 2
    $\begingroup$ Thanks for the answer! Just one question though. I can see why this cycle transformation is needed for month information since we need the ML algorithm to recognize that December and January are indeed close to each other. But does the same logic apply for year as well? For something like month that cycles within a certain range of numbers, this approach makes sense, but year progresses with cycling. $\endgroup$
    – cck3
    Commented Nov 3, 2017 at 6:19
  • $\begingroup$ You're right, my mistake. I updated my answer so that year is taken out from the list. The same logic does not apply to year as well. $\endgroup$
    – Mark White
    Commented Nov 3, 2017 at 13:44
  • $\begingroup$ I wonder how these can be applied when time/date are features - for example stackoverflow.com/questions/59653862/… $\endgroup$
    – Areza
    Commented Jan 16, 2020 at 22:06
  • 1
    $\begingroup$ @user4581first thing that comes to mind is maybe put the data in long format, where each row would be a combination of observation/encoded datetime/the value of that datetime (a, b, c, NaN, etc.) $\endgroup$
    – Mark White
    Commented Jan 16, 2020 at 22:15

I know I left a comment, but just to be explicit here is Feature-Engine's CylicalFeatures.

import pandas as pd
from sklearn.model_selection import train_test_split

from feature_engine.creation import CyclicalFeatures

df = pd.DataFrame({
    'day': [6, 7, 5, 3, 1, 2, 4],
    'months': [3, 7, 9, 12, 4, 6, 12],

cyclical = CyclicalFeatures(variables=None, drop_original=True)

X = cyclical.fit_transform(df)


      day_sin         day_cos          months_sin     months_cos
1    -0.78183         0.62349             1.0             0.0
2         0.0             1.0            -0.5        -0.86603
3    -0.97493       -0.222521            -1.0            -0.0
4     0.43388       -0.900969             0.0             1.0
5     0.78183         0.62349         0.86603            -0.5
6     0.97493       -0.222521             0.0            -1.0
7    -0.43388       -0.900969             0.0             1.0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.