I am working on a Machine Learning regression problem, with a data-set where I have data from a period of several years. From the "date" feature, I extracted the week number (0-53). Next I am doing 2 things:

1) One hot encoding: Splitting this categorical "week number" feature into 53 binary features, where each feature indicates whether the data points belong to that particular week number or not.

2) I am also using the cyclic variable (week number) as a continuous variable to predict my outcome. First I am converting this feature, however, to the distance from week 1 (so week 2 and 53 don't represent drastically different time points)

My question is, am I making this too complicated without increasing potential improvements in my model outcome? Does including the continuous variable actually provide my model with valuable information that is not already covered in the categorical feature extraction? Thank you in advance

  • 2
    $\begingroup$ My initial response would be to you a spline for the weekly trend. A simpler approach would be using a pair of Fourier terms to model the weekly periodicity (e.g. sin(2*pi*(1/52.15)*weekNumber) + cos(2*pi*(1/52.15)*weekNumber) ). Using both a 53-leveled factor as well as the week number in your model probably loses you way too many degrees of freedom. $\endgroup$ – usεr11852 Oct 14 '18 at 0:12
  • $\begingroup$ Thanks for your suggestions. I understand the concern for losing too many degrees of freedom. I learned earlier, however, that losing degrees of freedom isn't the biggest issue if I have a large enough data-set. In this case, I have 17,000 data points and 120 features even after the one hot encoding, so I would say this is still a "skinny" dataset. Do you think losing degrees of freedom should still be a big concern for me? $\endgroup$ – stats_nerd Oct 14 '18 at 2:11
  • $\begingroup$ Sorry missed your response. No, it is not a huge concern. That said using a spline is more natural and would allow determining the strength of the pattern more readily. $\endgroup$ – usεr11852 Mar 12 '20 at 20:48

I think what you are doing is unnecessary.

Encoding of categorical features heavily depends on model choice:

In the case of linear models and Nets.:

It is necessary to encode it as a continuous variable or one-hot encoding. In your example, I believe it is better to encode it as a continuous variable because dimensionality is relatively high. Some approaches:

  • Mean encoding(expanding one if you choose this option)
  • Encoding as cyclical, I leave a piece of code I use often in my projects:

In the case of tree-based models:

  • You can go with any method, but I would not recommend the one-hot-encoding approach again because of the dimensionality problem
  • Ordinal features generally work well. Sometimes mean encoding provides some improvement

In any case suggestion from @Ugur MULUK is helpful. Date features like "week of the month, day of the week, season, hour, a year or whether it is not a business day, etc." helps a lot.

Code I use to create cyclical feats.:

df['hour_sin'] = np.sin(2 * np.pi * df.DATETIME.dt.hour/23)
df['hour_cos'] = np.cos(2 * np.pi * df.DATETIME.dt.hour/23)
df['day_sin'] = np.sin(2 * np.pi * df.DATETIME.dt.dayofweek/6)
df['day_cos'] = np.cos(2 * np.pi * df.DATETIME.dt.dayofweek/6)

1) What you are doing is one-hot-encoding. That will give you a sparser dataset, but you do not need both at the same time.

2) You should leave categorical variables as categorical, what you are doing is dangerous. Numeric to categorical transformation would be acceptable however by digitization and grouping.

I do not know your machine learning task. However, I’d recommend you to get features like week of the month, day of the week, season, hour, year or whether it is not a business day, etc.

  • $\begingroup$ Thanks for your response and for reminding me of the process name. Could you please elaborate on why what I am doing right now is "dangerous"? And also could you please elaborate on what you mean exactly by "digitization and grouping"? The data-set already has features for month number, weekday, season and hour. However week of the month isn't one that was included, so I think I might incorporate that now. Thank you for the suggestion $\endgroup$ – stats_nerd Oct 14 '18 at 1:46
  • $\begingroup$ About the danger thing; think of a case such that you have a categorical feature that has values like “apple”, “spinach”, “grapefruit”, and so on. You could of course use encode those as categories 0, 1, 2, 3....,n. But if you give them as a numeric feature; would it make sense? Is apple spinach related linearly or even nonlinearly? I have not said you can’t but you shouldn’t since weeks are related in a time-series manner, but think carefully about it. Digitization is you make you make your numerıc features round up to some interval boundaries by grouping them in a reasonable sense. $\endgroup$ – Ugur MULUK Oct 14 '18 at 8:30
  • $\begingroup$ Think of your numeric feature has its 95% of values from 45 to 80, and rest are outliers. Cut the outliers by np.clip and make them 45 or 80. Than make your categorical feature as 45-to-50, 50-to-55,..... 75-to-80 as 0,1,2,3,4. Decide on the resolution by yourself, one method for this is np.digitize. I wrote all by the phone, sorry about the typos in the answers. $\endgroup$ – Ugur MULUK Oct 14 '18 at 8:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.