24
$\begingroup$

Working on regression problem I started to think about representation of "day of a week" feature. I wonder which approach would perform better:

  • one feature; value 1/7 for Monday; 2/7 for Tuesday...
  • 7 features: (1, 0, 0, 0, 0, 0, 0) for Monday; (0, 1, 0, 0, 0, 0, 0) for Tuesday...

It's hard to measure it due to network configuration differences. (Additional six features should be reflected in number of hidden nodes I believe.)

Number of all features is about 20. I use simple backprop to learn ordinary feed-forward neural network.

$\endgroup$
3

1 Answer 1

26
$\begingroup$

Your second representation is more traditional for categorical variables like day of week.

This is also known as creating dummy variables and is a widely used method for encoding categorical variables. If you used 1-7 encoding you're telling the model that days 4 and 5 are very similar, while days 1 and 7 are very dissimilar. In fact, days 1 and 7 are just as similar as days 4 and 5. The same logic holds up for 0-30 encoding for days of the month.

Day of the month is a little trickier, because while every week has the same 7 days, not every month has the same 30 days: some months have 31 days, and some months have 28 days. Since both weeks and months are cyclical, you could use fourier transformations to convert them to smooth linear variables.

For example (using R, my programming language of choice):

day_of_month = c(1:31, 1:28, 1:30)
day_of_year <- 1:length(day_of_month)
s = sin((2*pi)/30*day_of_month)
c = cos((2*pi)/30*day_of_month)
plot(day_of_month ~ day_of_year)
lines(15*s+15 ~ day_of_year, col='blue')
lines(15*c+15 ~ day_of_year, col='red')
legend(10, 30, c('raw', 'sin', 'cos'), c('black', 'blue', 'red'))

raw vs sin vs cosine

(I scaled the sine/cosine variables to be 0/30, rather than -1/1 so the graph looks better)

As you can see, while the raw "day of month variable" jumps back to zero at the end of each month, the sine and cosine transformations make a smooth transition that lets the model know days at the end of one month are be similar to days at the beginning of the next month.

You can add the rest of the fourier terms as follows:

for(i in 1:3){
  s = sin((2*pi)/30*day_of_month + 30 * i/4)
  c = cos((2*pi)/30*day_of_month + 30 * i/4)
  lines(15*s+15 ~ day_of_year, col='blue')
  lines(15*c+15 ~ day_of_year, col='red')
}
legend(10, 30, c('raw', 'sin', 'cos'), c('black', 'blue', 'red'))

Complete transforms

Each pair of sine/cosine waves makes a circle:

m <- lapply(1:4, function(i){
  as.matrix(
    data.frame(
    s = sin((2*pi)/30*day_of_month + 30 * i/4),
    c = cos((2*pi)/30*day_of_month + 30 * i/4)
    )
  )
})
m <- do.call(cbind, m)
pairs(m)

circle This page has a really handy explanation of how to manipulate sine and cosine waves.

$\endgroup$
5
  • $\begingroup$ Is there any specific reason to do so? I wonder how could it affect convergence. My second doubt is when a variable is still categorical - what about day of month? (0 - 30) $\endgroup$
    – Oepas Dost
    Dec 2, 2014 at 0:41
  • $\begingroup$ Same thing; use indicator variables. The first encoding induces a similarity measure that may not be appropriate; e.g., is Sunday really the most dissimilar day from Monday? That's what the encoding implies... $\endgroup$
    – Emre
    Dec 2, 2014 at 0:53
  • $\begingroup$ @OepasDost If my post answers your question, feel free to up-vote it and/or accept it by clicking the checkmark. $\endgroup$
    – Zach
    Dec 2, 2014 at 13:44
  • $\begingroup$ @Zach why would you consider day of week (that can be encoded from 0 to 6) as categorical and day of month as ordinal cyclical (and therefore use the Fourier transform). And not both of them as ordinal cyclical and therefore do a Fourier transform for day of week as well? $\endgroup$
    – zipp
    Nov 29, 2017 at 20:59
  • 1
    $\begingroup$ @zipp You could use a Fourier for day of the week too. In my experience, the primary value of day of week is the difference between weekdays and weekends, which is very simple to capture with dummy variables (or an indicator variable). $\endgroup$
    – Zach
    Nov 30, 2017 at 18:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.