14
$\begingroup$

How to encode the date and time of an event for a neural network?

I do not have a continuous time series, but some events with date and time, and I analyze some kind of interest. This interest differs between mornings and evenings, and differs between weekdays, and between summer and winter, and before Christmas and Easter and so on. And the events themselves have a strong non-uniform distribution over time (more at day than at night, some kinds more during the week, some more at the weekend).

I tried encoding it as Number of Week in year, as Weekday 1-7 and as Hour of day. But playing around with a sparse autoencoder gave me the impression that my data does not make any sense to a neural network, it could not even reproduce anything near the input even with a big hidden layer. Neither as categorial 0-1 nor as normalized values.

But searching for the encoding of time for a neural network mostly gives information about time series, so im a bit blindfolded by the forest but looking for the tree.

Of course I could look at the data and roughly categorize it more or less despotic. But the concept of Deep Learning seems to sweep away all the hand-crafted manual feature extraction. And the categorization would insert big jumps in a naturally continuous input variable.

My "natural encoding" in my brain is more like a fuzzy membership to some categories like "night", "morning", "weekday" and so on.

To make the whole thing more interesting, also the dependend variable contains those date/time data, but that is a different question.

EDIT: Somehow related to the cyclic kind of data are some recent questions, like

Which statistical tests are reasonable with this time of day data set?

$\endgroup$
7
$\begingroup$

I was looking for an answer to a similar problem and stumbled on this thread. The sinusoidal encoding idea is explored in this blog post:

Encoding cyclical continuous features - 24-hour time

Ian's answer fully addressed my needs, so I thought about posting it here for future memory.

$\endgroup$
1
1
$\begingroup$

You could try representing time as a big matrix, i.e. a 365 by 24, to represent the days of the year and hours of the day, and then "unroll" this into a 1 by 8760 vector. The time would then correspond to the position within this vector and the value at this position is the value at that time.

$\endgroup$
2
  • 2
    $\begingroup$ Have you tried and succeeded with an encoding like that? I would be surprised if a Neural Net would "learn" the exact positions of sunday morning in this encoding. But to surprise the naive is one of the strengths of neural nets, so I would not bet my Scotch against it. ;-) $\endgroup$ Jan 29 '16 at 22:18
  • $\begingroup$ This could be useful if you wanted to detect events cyclical by the same hour in various years, but it seems to me correlation will be very weak. I can see a higher likelihood of a correlation between the same hour each week, or the same hour each day, for most time series data. $\endgroup$ Nov 16 '16 at 18:48
1
$\begingroup$

I would suggest creating multiple input features from the time series using relationships you know (or believe) to exist already in the data. For example, you state that the target output will vary:

between mornings and evenings, and differs between weekdays, and between summer and winter,...

So why not create a set of features that describes each of these 'cycles'. This may help to tease out both micro and macro variations rather than a single feature that describes all.

For example...

If you have trend whereby something of interest occurs around midday each day, then create a feature from $1..24$ that describes the hours in the day. Now the network will learn to trigger at around 12. Compare this to the case where you have this same data encoded as hours in a week $1..168$. Now the network has to try an learn to trigger on $12,36,60...$ which is significantly more complex.

$\endgroup$
2
  • $\begingroup$ Yeah, this was my first idea too. But the circular concept of time (23:59 is followed by 00:00) is then hidden, and another thing that bothers me is the jump between the seemingly whole numbers - an event at 09:55 is very similar to 10:05, but in the morning 06:10 is very different from 06:55. I could imagine to search centers of the time (optics or the like?) and then measure and give the distance to those centers. So 04:30 a.m. is deepest night, whereas 05:30 is more "morningly", but completely not like evening. $\endgroup$ Jan 29 '16 at 22:15
  • 1
    $\begingroup$ Well in that case you could try encoding as a sinusoid or cosine, or in fact both. $\endgroup$ Jan 30 '16 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.