15
$\begingroup$

I'm not used to using variables in the date format in R. I'm just wondering if it is possible to add a date variable as an explanatory variable in a linear regression model. If it's possible, how can we interpret the coefficient? Is it the effect of one day on the outcome variable?

See my gist with an example what I'm trying to do.

$\endgroup$

migrated from stackoverflow.com Jul 29 '13 at 13:50

This question came from our site for professional and enthusiast programmers.

  • 2
    $\begingroup$ A date can be converted to a number. $\endgroup$ – user2636 Jul 10 '13 at 13:40
  • $\begingroup$ my impression is that R does it automatically $\endgroup$ – PAC Jul 10 '13 at 13:42
  • 3
    $\begingroup$ But the resulting numbers are often huge, which can lead to problems. Better to convert yourself, e.g. to time steps (hours or days or...) since beginning of measurements. That also makes it easier to interpret the intercept. $\endgroup$ – Roland Jul 10 '13 at 13:43
  • 3
    $\begingroup$ Convert to factor (to get day fixed effects) or convert to numeric and rescale so value of first day is 0 to get a linear effect of days since origin. $\endgroup$ – Thomas Jul 10 '13 at 13:58
  • 1
    $\begingroup$ That's a real good question. I think that it is both, a statistical question and a programming question. The programming question is how does R deals with dates when we put dates as an explanatory variable in a regression model and the statistical question is about the exact interpretation of the coefficient. $\endgroup$ – PAC Jul 11 '13 at 10:18
17
$\begingroup$

Building on earlier comments on Stack Overflow:

Yes, it makes sense. Here I address the general question and am happy to let R experts fill in the crucial details. In my view, as this is now on Cross-Validated, we should not focus too narrowly on the poster's favourite software, important though that is for like-minded people.

Dates in any software if not numeric can be converted to numeric variables, expressed in years, days, milliseconds or whatever since some time origin. The coefficient associated with each date has denominator units which are whatever the units of the date are. The numerator units depend on those of the response or dependent variable. (Non-identity link functions complicate this, naturally.)

However, it usually makes most sense when dates are shifted to an origin that makes sense for the study. Usually, but not necessarily, the origin should be a date within the time period of study or very close to it.

Perhaps the simplest case is linear regression on a date variable in years. Here a regression of some response on date expressed as dates like 2000 or 2010 implies an intercept which is the value of response in year 0. Setting aside the calendrical detail that there was no such year, such an intercept is often absurdly large positive or negative, which is logical but a distraction in interpretation and presentation (even to well-informed audiences).

In a real example from working with undergraduate students, the number of cyclones per year in a certain area was increasing slightly with the date and a linear trend looked a reasonable first stab. The intercept from regression was a large negative number, which caused much puzzlement until it was realised that this was, as always, an extrapolation to year 0. Shifting the origin to 2000 produced better results. (Actually, a Poisson regression ensuring positive predictions was even better, but that's a different story.)

Regressing on date - 2000 or whatever is thus a good idea. The substantive details of a study often indicate a good base date, i.e. a new origin.

The use of other models and/or other predictors doesn't undermine this principle; it just obscures it.

It is also a good idea to graph results using whatever dates are easiest to think about. These may be the original dates; that's not a contradiction, as it is just the same principle of using whatever is easiest to think about.

A little thought shows that the principle is much more general. We are often better off with (age - 20) or some such, to avoid logical but awkward predictions for age 0.

EDIT 21 March 2019 (original 29 Jul 2013): These arguments have been discussed in a Stata context in Cox, N.J. 2015. Species of origin. Stata Journal 15: 574-587 see here

EDIT 2 also 4 Dec 2015 @whuber in comments raises also the important issue of numerical precision. Often the time units are fine and the resulting dates or date-times can be very large, raising important issues for sums of squares, and so on and so forth. He raises an example from R. To that we can add (e.g.) that date-times in Stata are milliseconds since the start of 1960. This problem is not at all specific to dates, as it can arise generally with numbers that are very big or very small, but it is worth flagging too.

$\endgroup$
  • 1
    $\begingroup$ Econometrically speaking, date is often used as a proxy for an immeasurable variable, or simply data you can not easily obtain. This can be seen in the increased sales rate of a certain product from a new company over time as brand recognition increases. Since you most likely don't have a metric for brand recognition, date can be used as a proxy. This would give your other regressors "more realistic" coefficients.**TL:DR** you should be careful using date in your regression without thinking about what unmeasured factors date may be correlated with that would affect your independent variable $\endgroup$ – scott Jul 29 '13 at 17:55
  • 1
    $\begingroup$ Good advice. I'd conjecture that (a function of) calendar date is typically a proxy for some process in time that is difficult to capture otherwise, so the point extends way beyond econometrics. $\endgroup$ – Nick Cox Jul 29 '13 at 18:02
  • 1
    $\begingroup$ I am as fond of sines and cosines as anyone, but what sampling of problems across disciplines leads to that judgement? $\endgroup$ – Nick Cox Jul 29 '13 at 20:12
  • 1
    $\begingroup$ If interested see stata-journal.com/sjsearch.html?choice=keyword&q=season for links to some of my work on seasonality. $\endgroup$ – Nick Cox Jul 29 '13 at 21:53
  • 3
    $\begingroup$ +1 The comments about shifting the origin are important. I recently rewrote part of a popular R regression package that failed badly with date regressors because it did not standardize them internally. Since the default date representation is in seconds since 1970, recent dates are in the billions and the squaring involved in the formulas obliterates most or all of the precision in the results when other regressors have values near $1$ or smaller. Wiser choices of date origin (and units of measurement--decimal years almost always work well) tend to avoid such problems. $\endgroup$ – whuber Dec 4 '15 at 15:33
5
$\begingroup$

As been said above, with suitable scaling, dates are great regressors. Time effects are less likely to be linear than even the typical covariates, so I almost always use regression splines in time. Some complex time trends require many knots (e.g., 7 or more) to fit. Restricted cubic splines (natural splines) provide safer linear extrapolation beyond the end of the observed times, although extrapolation is seldom completely safe.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.