Optimal construction of day feature in neural networks

Question

Working on regression problem I started to think about representation of "day of a week" feature. I wonder which approach would perform better:

• one feature; value 1/7 for Monday; 2/7 for Tuesday...
• 7 features: (1, 0, 0, 0, 0, 0, 0) for Monday; (0, 1, 0, 0, 0, 0, 0) for Tuesday...

It's hard to measure it due to network configuration differences. (Additional six features should be reflected in number of hidden nodes I believe.)

Number of all features is about 20. I use simple backprop to learn ordinary feed-forward neural network.

Answer 1

Your second representation is more traditional for categorical variables like day of week.

This is also known as creating dummy variables and is a widely used method for encoding categorical variables. If you used 1-7 encoding you're telling the model that days 4 and 5 are very similar, while days 1 and 7 are very dissimilar. In fact, days 1 and 7 are just as similar as days 4 and 5. The same logic holds up for 0-30 encoding for days of the month.

Day of the month is a little trickier, because while every week has the same 7 days, not every month has the same 30 days: some months have 31 days, and some months have 28 days. Since both weeks and months are cyclical, you could use fourier transformations to convert them to smooth linear variables.

For example (using R, my programming language of choice):

day_of_month = c(1:31, 1:28, 1:30)
day_of_year <- 1:length(day_of_month)
s = sin((2*pi)/30*day_of_month)
c = cos((2*pi)/30*day_of_month)
plot(day_of_month ~ day_of_year)
lines(15*s+15 ~ day_of_year, col='blue')
lines(15*c+15 ~ day_of_year, col='red')
legend(10, 30, c('raw', 'sin', 'cos'), c('black', 'blue', 'red'))

(I scaled the sine/cosine variables to be 0/30, rather than -1/1 so the graph looks better)

As you can see, while the raw "day of month variable" jumps back to zero at the end of each month, the sine and cosine transformations make a smooth transition that lets the model know days at the end of one month are be similar to days at the beginning of the next month.

You can add the rest of the fourier terms as follows:

for(i in 1:3){
  s = sin((2*pi)/30*day_of_month + 30 * i/4)
  c = cos((2*pi)/30*day_of_month + 30 * i/4)
  lines(15*s+15 ~ day_of_year, col='blue')
  lines(15*c+15 ~ day_of_year, col='red')
}
legend(10, 30, c('raw', 'sin', 'cos'), c('black', 'blue', 'red'))

Each pair of sine/cosine waves makes a circle:

m <- lapply(1:4, function(i){
  as.matrix(
    data.frame(
    s = sin((2*pi)/30*day_of_month + 30 * i/4),
    c = cos((2*pi)/30*day_of_month + 30 * i/4)
    )
  )
})
m <- do.call(cbind, m)
pairs(m)

This page has a really handy explanation of how to manipulate sine and cosine waves.