Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a fairly large dataset with a few fields containing time-related data. This data comes in various shapes and sizes, but most of it can be parsed and rephrased in more appropriate formats for human-reading.

Formats range from 2010-01-01 to less obvious like 2008-52 (the second number is the number of editions a magazine had -- in this case, 1 for each week of the year).

Now I trying to transform it features for crunching. I first I thought it make sense to calculate the "age", i.e., the time difference between now() and the timestamp I have in my data, and later normalize it. I know this works (computationally speaking), but I don't know if is the best approach or produce anything useful.

Is this approach useful? Is there a better way to handle "date" features?

share|improve this question

3 Answers 3

up vote 3 down vote accepted

What you suggest -- now -- is an absolute baseline date.

You could also use a base date that repeats on a regular basis. For example, if you're looking at US shopping patterns, the number of days before Christmas would be useful. Or if you were looking at international soccer, perhaps the number of days before or after the World Cup.

You could also use a base date that is based on a particular event in each time series. For example, a person's age (number of days since birth), or the number of years a person has had a driver's license, or how long since they were diagnosed with a particular illness (or started treatment).

This kind of date handling allows you to line up the various time series in a more meaningful way. The important thing is not turning a date into a computable object but turning it into a meaningful measurement -- that is a feature.

share|improve this answer
    
It makes sense to use a more significant baseline, maybe now is just too arbitrary. But I'd still be using a "age" feature, where I'd have to calculate the difference between to dates. I wonder if there is a better way, maybe label different "age" intervals, creating a new feature? But I don't know how to evaluate if this approach is better or worse than plain "ages" –  lsdr Jul 14 at 1:48
    
Why don't you want to calculate the difference between dates? You can also include an age^2 feature or even a unit less number like the age that year divided by the median age that year. Binning ages into intervals will throw away information and us usually not a good idea. –  Wayne Jul 14 at 11:46
    
It's not that I don't want to, I'm just trying to be a little critical to avoid traps. It seems the most straightforward approach. And using a more significant baseline it looks like what I need, anyways. –  lsdr Jul 14 at 12:03

It depends on what aspects of your data (you think) matter.

Absolute baselines are often a very reasonable choice: for example, people's ages (now minus thier date of birth) are reasonable proxies for all sorts of things, including spending habits, musical tastes, and health status. I would imagine that this also holds for companies (older company ==> more established market?) and maybe even certain types of objects too (older cars are either very valuable or worthless).

However, when using the current date as a baseline, you need to think carefully about the difference between the age of the data (i.e., when it was collected) and the ages of the subjects. For example, data collected in 1974 about the habits of 10 year olds probably indicates the habits of 10 year olds, not their current 40 year old selves.

share|improve this answer
    
I agree with both you and @Wayne regarding the baseline, I have to pick a reasonable one, not just pick now and go with it, but I wonder, if I choose to create labels grouping intervals of date instead, how could I evaluate it this is a better alternative? –  lsdr Jul 14 at 1:56
1  
I think the conventional wisdom is that binning should be avoided--you're throwing away potentially useful information. On the other hand, some phenomena really do have hard edges. For example, 65, 67, and 70.5 are important ages for retirement benefits in the US and it'd be silly to insist that your analysis "rediscovers" those values. –  Matt Krause Jul 30 at 20:30

You can take direct no. of days by using

> Sys.Date()- as.Date("2012-03-12")# calculate the days
Time difference of 847 days

or in week format

> round(as.numeric((Sys.Date()- as.Date("2012-03-12"))/7))# calculate the Weeks
[1] 121

You can take groups depends upon week or year like 0 to 5 weeks/years then You can normalized the data.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.