Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I often deal with reasonable sized amount of time series data, 50-200 million doubles with associated time stamps and would like to visualize them dynamically.

Is there existing software to do this effectively? How about libraries and data formats? Zoom-cache is one example of library focusing on large time series. In Zoom-cache the data is summarized at several resolutions to make it easier to view at different resolutions.

Edit: Also if there is anywhere else I should ask this question or seek an answer please let me know.

share|improve this question
Although I have no idea about how it will scale with such huge number of data points, you might want to take a look at Cubism.js. – chl Jan 17 '13 at 9:27
I took a look at Cubism, which looks good for incremental updating graphs from serial streams of data, but with my short search I did not see anything about caching multiple resolutions or summary data to speed up large data sets. Thats for the pointer though it looks like a cool library. – Davorak Jan 17 '13 at 23:39

There are a set of research tools called TimeSearcher 1--3 that provide some examples of how to deal with large time-series datasets. Below are some example images from TimeSearcher 2 and 3.

TimeSearcher 2

TimeSearcher 3

share|improve this answer

Sorry for self-ad, but Thunderplot ( would be good candidate for it. I wrote it exactly for interactive plotting of large datasets. I didn't test it with 200KK rows datasets, but it works fine with ~10KK rows. Also there's "read every Nth row" feature, so you can reduce amount of data to store/visualize. I can send you the registration key in exchange to one of those large datasets. :-)

thunderplot screenshot

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.