15
$\begingroup$

How to encode the date and time of an event for a neural network?

I do not have a continuous time series, but some events with date and time, and I analyze some kind of interest. This interest differs between mornings and evenings, and differs between weekdays, and between summer and winter, and before Christmas and Easter and so on. And the events themselves have a strong non-uniform distribution over time (more at day than at night, some kinds more during the week, some more at the weekend).

I tried encoding it as Number of Week in year, as Weekday 1-7 and as Hour of day. But playing around with a sparse autoencoder gave me the impression that my data does not make any sense to a neural network, it could not even reproduce anything near the input even with a big hidden layer. Neither as categorial 0-1 nor as normalized values.

But searching for the encoding of time for a neural network mostly gives information about time series, so im a bit blindfolded by the forest but looking for the tree.

Of course I could look at the data and roughly categorize it more or less despotic. But the concept of Deep Learning seems to sweep away all the hand-crafted manual feature extraction. And the categorization would insert big jumps in a naturally continuous input variable.

My "natural encoding" in my brain is more like a fuzzy membership to some categories like "night", "morning", "weekday" and so on.

To make the whole thing more interesting, also the dependend variable contains those date/time data, but that is a different question.

EDIT: Somehow related to the cyclic kind of data are some recent questions, like

Which statistical tests are reasonable with this time of day data set?

$\endgroup$

3 Answers 3

8
$\begingroup$

I was looking for an answer to a similar problem and stumbled on this thread. The sinusoidal encoding idea is explored in this blog post:

Encoding cyclical continuous features - 24-hour time

Ian's answer fully addressed my needs, so I thought about posting it here for future memory.

$\endgroup$
1
1
$\begingroup$

You could try representing time as a big matrix, i.e. a 365 by 24, to represent the days of the year and hours of the day, and then "unroll" this into a 1 by 8760 vector. The time would then correspond to the position within this vector and the value at this position is the value at that time.

$\endgroup$
3
  • 2
    $\begingroup$ Have you tried and succeeded with an encoding like that? I would be surprised if a Neural Net would "learn" the exact positions of sunday morning in this encoding. But to surprise the naive is one of the strengths of neural nets, so I would not bet my Scotch against it. ;-) $\endgroup$ Commented Jan 29, 2016 at 22:18
  • $\begingroup$ This could be useful if you wanted to detect events cyclical by the same hour in various years, but it seems to me correlation will be very weak. I can see a higher likelihood of a correlation between the same hour each week, or the same hour each day, for most time series data. $\endgroup$ Commented Nov 16, 2016 at 18:48
  • $\begingroup$ I mean, it could work, but this representation completely disregards the order of days in the year and hours in the day, so the network wouldn’t be able to reuse knowledge it had learned about a certain time and day for the same time the next day until it had learned the relationships between those two points in time. If you instead use a continuous representation, those two time points will probably have very similar representations so the relationship will be much more obvious, so I assume that you will need less training data and that training will go quicker too. $\endgroup$ Commented Jan 22 at 17:02
1
$\begingroup$

I would suggest creating multiple input features from the time series using relationships you know (or believe) to exist already in the data. For example, you state that the target output will vary:

between mornings and evenings, and differs between weekdays, and between summer and winter,...

So why not create a set of features that describes each of these 'cycles'. This may help to tease out both micro and macro variations rather than a single feature that describes all.

For example...

If you have trend whereby something of interest occurs around midday each day, then create a feature from $1..24$ that describes the hours in the day. Now the network will learn to trigger at around 12. Compare this to the case where you have this same data encoded as hours in a week $1..168$. Now the network has to try an learn to trigger on $12,36,60...$ which is significantly more complex.

$\endgroup$
2
  • $\begingroup$ Yeah, this was my first idea too. But the circular concept of time (23:59 is followed by 00:00) is then hidden, and another thing that bothers me is the jump between the seemingly whole numbers - an event at 09:55 is very similar to 10:05, but in the morning 06:10 is very different from 06:55. I could imagine to search centers of the time (optics or the like?) and then measure and give the distance to those centers. So 04:30 a.m. is deepest night, whereas 05:30 is more "morningly", but completely not like evening. $\endgroup$ Commented Jan 29, 2016 at 22:15
  • 1
    $\begingroup$ Well in that case you could try encoding as a sinusoid or cosine, or in fact both. $\endgroup$ Commented Jan 30, 2016 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.