Skip to main content
laid out the summary table so the later columns were lined up as well
Source Link

Please pardon my ignorance, but...

I keep finding myself in a situation, where I'm faced with a bunch of new data I managed to find. This data usually looks something like this:

Date     Number1  Number2 Category1  Category1  Category2
20120125      11      101        Dog      Brown
20120126      21       90        Cat      Black
20120126      31      134        Cat      Brown
(...)

Usually at first glance I can't really tell if there are any trends here. The correlations between the various columns may not be very significant, but I would be delighted if I didn't have to manually create a plot for every possible combination of columns/categories.

Is there a tool out there which would accept a table of data along with information which columns should be treated as numbers, dates and categories and then proceed to plot:

  • correlations between each two numerical columns
  • correlations between each two numerical columns, with separate trend lines for each category
  • each number column as a time series,
  • each number column as a time series, separated by category,
  • etc.

In the end this would generate a large number of plots, most of which would show only noise. Ideally, the tool could score the plots by correlation and in the end display a slideshow starting with the highest scoring plots. This would be a very imperfect, but useful first glance at the dataset.

So? Is there a tool everyone uses for this and I just don't know about it, or is this something we need to make?

Please pardon my ignorance, but...

I keep finding myself in a situation, where I'm faced with a bunch of new data I managed to find. This data usually looks something like this:

Date    Number1  Number2 Category1    Category2
20120125    11   101 Dog Brown
20120126    21   90 Cat Black
20120126    31   134 Cat Brown
(...)

Usually at first glance I can't really tell if there are any trends here. The correlations between the various columns may not be very significant, but I would be delighted if I didn't have to manually create a plot for every possible combination of columns/categories.

Is there a tool out there which would accept a table of data along with information which columns should be treated as numbers, dates and categories and then proceed to plot:

  • correlations between each two numerical columns
  • correlations between each two numerical columns, with separate trend lines for each category
  • each number column as a time series,
  • each number column as a time series, separated by category,
  • etc.

In the end this would generate a large number of plots, most of which would show only noise. Ideally, the tool could score the plots by correlation and in the end display a slideshow starting with the highest scoring plots. This would be a very imperfect, but useful first glance at the dataset.

So? Is there a tool everyone uses for this and I just don't know about it, or is this something we need to make?

Please pardon my ignorance, but...

I keep finding myself in a situation, where I'm faced with a bunch of new data I managed to find. This data usually looks something like this:

Date     Number1  Number2  Category1  Category2
20120125      11      101        Dog      Brown
20120126      21       90        Cat      Black
20120126      31      134        Cat      Brown
(...)

Usually at first glance I can't really tell if there are any trends here. The correlations between the various columns may not be very significant, but I would be delighted if I didn't have to manually create a plot for every possible combination of columns/categories.

Is there a tool out there which would accept a table of data along with information which columns should be treated as numbers, dates and categories and then proceed to plot:

  • correlations between each two numerical columns
  • correlations between each two numerical columns, with separate trend lines for each category
  • each number column as a time series,
  • each number column as a time series, separated by category,
  • etc.

In the end this would generate a large number of plots, most of which would show only noise. Ideally, the tool could score the plots by correlation and in the end display a slideshow starting with the highest scoring plots. This would be a very imperfect, but useful first glance at the dataset.

So? Is there a tool everyone uses for this and I just don't know about it, or is this something we need to make?

edited tags
Link
onestop
  • 18k
  • 2
  • 63
  • 91
Tweeted twitter.com/#!/StackStats/status/162904934222802944
Source Link

First quick glance at a dataset

Please pardon my ignorance, but...

I keep finding myself in a situation, where I'm faced with a bunch of new data I managed to find. This data usually looks something like this:

Date    Number1  Number2 Category1    Category2
20120125    11   101 Dog Brown
20120126    21   90 Cat Black
20120126    31   134 Cat Brown
(...)

Usually at first glance I can't really tell if there are any trends here. The correlations between the various columns may not be very significant, but I would be delighted if I didn't have to manually create a plot for every possible combination of columns/categories.

Is there a tool out there which would accept a table of data along with information which columns should be treated as numbers, dates and categories and then proceed to plot:

  • correlations between each two numerical columns
  • correlations between each two numerical columns, with separate trend lines for each category
  • each number column as a time series,
  • each number column as a time series, separated by category,
  • etc.

In the end this would generate a large number of plots, most of which would show only noise. Ideally, the tool could score the plots by correlation and in the end display a slideshow starting with the highest scoring plots. This would be a very imperfect, but useful first glance at the dataset.

So? Is there a tool everyone uses for this and I just don't know about it, or is this something we need to make?