16
$\begingroup$

I am trying to fit a linear model using "hour of the day" as parameter. What I'm struggling with, is, that I've found two possible solutions on how to handle this:

  1. Dummy encoding for every hour of the day

  2. Transform hours into cyclic variable

I don't quite understand the use cases of both approaches and thus I am not certain which one will lead to a better outcome.

The Data I'm using is from this Kaggle challenge. The goal is to predict nyc taxi fares. Given attributes are pickup and dropoff coordinates, pickup datetime, passenger count and the fare amount. I extracted the hour of the day to take possible congestions into consideration and am trying to implement it into my model. I should also probably mention that I'm pretty inexperienced.

$\endgroup$
5
  • 2
    $\begingroup$ Does your data involve more than 1 day ? Please explain the study design and your research questions $\endgroup$ Jul 20 '20 at 8:50
  • 2
    $\begingroup$ can you briefly explain what kind of data you have and what you want to infer from it? $\endgroup$
    – StupidWolf
    Jul 20 '20 at 8:57
  • $\begingroup$ A hierarchical model would allow you to have the best of both worlds, by having discrete effects for each hour which are correlated based on proximity. However, in the large data limit, it will converge to the answer given by dummy encoding. $\endgroup$ Jul 20 '20 at 20:34
  • $\begingroup$ For a robust, flexible, parsimonious method, consider using circular splines. $\endgroup$
    – whuber
    Jul 21 '20 at 12:23
  • 1
    $\begingroup$ @whuber This is a good suggestion! But it appears that we have almost no questions or answers about circular splines! stats.stackexchange.com/search?q=circular+splines $\endgroup$
    – Sycorax
    Jul 23 '20 at 19:19
15
$\begingroup$

Dummy encoding would destroy any proximity measure (and ordering) among hours. For example, the distance between 1 PM and 9 PM would be the same as the distance between 1 PM and 1 AM. It'd be harder to say something like around 1 PM.

Even leaving them as is, e.g. numbers in 0-23, would be a better approach than dummy encoding in my opinion. But, this way has a catch as well: 00:01 and 23:59 would be seen very distant but actually they're not. To remedy this, your second listed approach, i.e. cyclic variables, is used. Cyclic variables map hours onto a circle (like a 24-h mechanical clock) so that the ML algorithm can see the neighbours of individual hours.

$\endgroup$
1
  • $\begingroup$ +1 but the continuous modeling will still be a challenge in terms of picking up things like rush hour congestion. Time of day will have at least a couple of peaks a day in terms of its contribution to outcome, and there will be an important interaction with day of week (workdays versus weekends and holidays). $\endgroup$
    – EdM
    Jul 20 '20 at 14:42
12
$\begingroup$

+1 to gunes' answer. Dummy coding will indeed disregard the distance between time points - the responses between two time points 1 hour apart will be more alike than between two time points 3 hours apart, and dummy coding completely discards this piece of information.

Dummy encoding fits a step-like time dependency: the response is flat for one hour, and then it suddenly jumps (and the jump is unconstrained except for what the data tells us - this is a consequence of the lack of proximity modeled). Both aspects are ecologically extremely doubtful:

enter image description here

Here is an additional aspect. If you bucketize your day into 24 hours, then you need to fit 23 parameters in addition to the intercept. This is a lot, and you will need a huge amount of data to reliably fit this without running afoul of the bias-variance tradeoff.

An alternative would be to use a Fourier-type model with harmonics. For instance, assume your observation timestamp $t$ corresponds to a time of day $\tau(t)$ (so when going from $t$ to $\tau(t)$, we simply drop the day, month and year information from $t$). Then you can transform the time impact into sines and cosines:

$$ \sin\big(2\pi k\frac{\tau(t)}{24}\big), \quad\cos\big(2\pi k\frac{\tau(t)}{24}\big). $$

A simple model would go up to $k=3$:

$$ y_t = \beta_0+\sum_{k=1}^3 \beta_k\sin\big(2\pi k\frac{\tau(t)}{24}\big) + \sum_{k=1}^3\gamma_k\cos\big(2\pi k\frac{\tau(t)}{24}\big) + \text{other covariates}+\epsilon_t. $$

This already gives you a lot of flexibility at the cost of fitting only 6 parameters, so your model will be far more stable. Also, you will get neither the constant response within an hour, nor the abrupt steps when a new hour starts. Here are some random examples of time courses this can fit:

harmonics

Of course, regardless of what choice you make, you should think about including any additional pieces of information you know (e.g., if all theaters and cinemas start or finish their shows at the same point in time, then mark this with a dummy, because then you will get a sharp step change, at least in the relevant districts). Also, the time response will certainly differ between weekdays and weekends, and likely also between Fridays and other weekdays, so include interactions between your time model and the day of week. Or look into models for to address this.

R code for my plots:

par(mai=c(.8,.1,.1,.1))
plot(c(0,24),c(0,1),yaxt="n",xlab="Hour",ylab="")
lines(c(0,rep(1:23,each=2),24),rep(runif(24),each=2))

tau <- seq(0,24,by=.001)
mm <- cbind(1,sin(2*pi*1*tau/24),sin(2*pi*2*tau/24),sin(2*pi*3*tau/24),cos(2*pi*1*tau/24),cos(2*pi*2*tau/24),cos(2*pi*3*tau/24))
par(mai=c(.8,.1,.1,.1),mfrow=c(3,2))
for ( ii in 1:6 ) plot(tau,(mm%*%runif(7,-1,1))[,1],yaxt="n",xlab="Hour",ylab="",type="l")
$\endgroup$
2
$\begingroup$

For a time series regression, simply adding hourly dummies $D_h, h = 0,\cdots, 23$, is the natural thing to do in most cases, i.e. fit the model $$ y_t = \beta_0 D_0 + \cdots + \beta_{23}D_{23} + \mbox{ other covariates } + \epsilon_t. $$ As a modeler, you're simply saying that the dependent variable $y_t$ has a hourly-dependent average $\beta_h$ at hour $h$, plus the effect from other covariates. Any hourly (additive) seasonality in the data would be picked up by this regression. (Alternatively, seaonsality can be modeled multiplicatively by, say, a SARMAX-type model.)

Transforming the data by some arbitrary periodic function (sin/cos/etc) is not really appropriate. For example, say you fit the model $$ y_t = \sum_{h=0}^{23} \beta_{h}\sin(2 \pi \frac{h(t)}{24}) + \mbox{ other covariates } + \epsilon_t, $$ where $h(t) = 12$ if observation $y_t$ is sampled at the 12th hour of the day (for example). Then you're imposing a peak at hour $h = 6$ (or whenever, by transforming the sine function) on the data, arbitrarily.

$\endgroup$
5
  • $\begingroup$ Surely there's a value judgement to be made here, depending on how important it is to avoid a potential discontinuity in the output of your model? $\endgroup$
    – user3490
    Jul 20 '20 at 16:57
  • $\begingroup$ @user3490 "...depending on how important it is to avoid a potential discontinuity in the output of your model..."---if the data says there is a jump ("discontinuity") between hour 11 and hour 12, avoiding/suppressing it artificially probably leads to misleading results. $\endgroup$
    – Michael
    Jul 20 '20 at 18:55
  • $\begingroup$ If you're doing linear regression, then it's not true that any seasonality will be detected. Only seasonality with a linear component will be detected. $\endgroup$ Jul 20 '20 at 22:04
  • 2
    $\begingroup$ I don't understand the trigonometric model you propose (and dismiss). The trig terms have no relationship to the time point $t$, and the initial term $\sum_{h=0}^{23} \beta_{h}\sin(2 \pi \frac{h}{24})$ functions like a constant. You are right that this is not useful, but it's a straw man. Sorry, but -1. $\endgroup$ Jul 21 '20 at 9:51
  • $\begingroup$ I am predicting number of car accidents per hour via weather conditions (categorical) and datetime (weekday, month, hour). I have made dummy variables for weekday since there are only 6 (2-7) and don't know what to do for month (1-12) and hour (0-23). Should I create dummy variables n-1 for months (2-12) then dummy variables for hour (1-23)? $\endgroup$ Dec 17 '20 at 11:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.