8
$\begingroup$

Suppose my CSV file has time and various columns of data vs time.

I want to be able to automate plotting. To a file at least; multiple formats or to the screen as well is a plus.

There will be lots of plots. Multiple plots on same axes, e.g. like a spreadsheet scatterplot with multiple columns. Changing the plot commands should be possible for a non-specialist. It should be possible to have some artistic control, like colors, points, lines, or both, maybe symbol.

Here's what I have considered:

spreadsheets like OpenOffice or Excel: can't automate easily. Good on looks, artistic control, and selecting columns from CSV to plot. Bad on automation, and time to plot.

R: Easy to read multi-column csv into dataframe, but not so easy to generate required plots. Easy for single time series but haven't seen any simple examples for plotting multiple columns of data onto the same X and Y axes as a spreadsheet scatterplot will do.

Gnuplot: May require reformatting data. Not so intuitive for reading multi-column CSV for scatterplot.

Any others I should be considering?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ I think if you have a strategy for abstracting your data operations, implementing in R wont be an issue. $\endgroup$
    – suncoolsu
    Commented Aug 19, 2011 at 3:50
  • 1
    $\begingroup$ Multiple columns on the same X and Y in R? matplot and you're done. $\endgroup$
    – user88
    Commented Aug 19, 2011 at 10:00
  • 3
    $\begingroup$ It seems to me all three of your assertions are wrong: Excel's default plots are ugly, R is well-known to produce nice graphics (and so-called spaghetti plots are easily handled within R, as @mbq said), Gnuplot has strictly no problem with multiple columns input data. But the real question is about automation although you didn't specify what you're expecting: Are you seeking for a script that might be launched in a given directory with a lot of csv files, do you want to build a minimalist GUI, should this be run locally or on a website, etc.? $\endgroup$
    – chl
    Commented Aug 19, 2011 at 11:31
  • $\begingroup$ @mbq <matplot looks interesting. chi Actually no one mentioned iteratively using the R functions points(), lines(), and legend() which also would do it. I had forgotten about these. While the excel graphs have ugly defaults, R will do nothing without the magic words and then let you shoot yourself in the foot just fine. A human non-coder can manually change and fix excel/OO charts to suit their needs. Or an inhumane non-coder can demand that other humans waste their time on manually importing data and fixing each spreadsheet graph to suit their ever changing needs. $\endgroup$
    – Paul
    Commented Aug 20, 2011 at 5:06
  • $\begingroup$ @user87 Although negatively worded, my preceding comment was not a critic, rather a request for clarification about what you really intend to do. There're many ways to automate things in R, including reporting (e.g., brew). $\endgroup$
    – chl
    Commented Aug 20, 2011 at 11:15

3 Answers 3

Reset to default
7
$\begingroup$

Python with MatplotLib. Python is pretty good at manipulating csv files.

but haven't seen any simple examples for plotting multiple columns of data onto the same X and Y axes as a spreadsheet scatterplot will do.

Have you explored ggplot2? You can keep adding series to a plot using ggplot2. It also has a very good facet plotting feature.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I selected this answer because I was looking for a "new" direction that I had not mentioned. I wonder if you have tried the entire Enthought bundle, which includes MatplotLib and a bunch of other analysis libs for python? Besides being a bit pricy for a distribution of free software, their FAQ mentions installing from their shell script. I wonder if they keep the system package manager clobbering the bundle and vice-versa. $\endgroup$
    – Paul
    Commented Aug 20, 2011 at 5:19
  • $\begingroup$ @user87 There's an academic version, free of charge. Also, Enthought recently released EPD Free (which includes a dedicated package manager), although it does not include all those nice packages (like Mayavi or Chaco) that are present in the full distrib. $\endgroup$
    – chl
    Commented Aug 20, 2011 at 11:19
6
$\begingroup$

As @suncoolsu observed the main thing is strategy for abstracting data operations. First prepare template for each graph you intend to produce. This means definining:

  1. Data for the graph
  2. Artistic details

Now you need the program (software package) which takes as the input the data and the artistic details and outputs the graph in your preferred format.

Data for the graph will probably be not of the format you have data in the csv file. So you need the program which reads the data from csv file and prepares it for plotting.

Finaly you will need a program which coordinates aforementioned processes: data preparation and graphs.

If you work with Unix based systems such combination of different program is very common, so there exist multiple choices for the data manipulation and coordination programs. All the scripting languages (bash, python, perl, ruby) will be able to perform these tasks. For producing graphs you need more specialized software, such as gnuplot, or specialized libraries of scripting languages. Although I mentioned Unix, you can perform these operations on Windows too.

Instead of scripting languages you can write dedicated programs in C, C++, Java, .NET or any other programming language you prefer. It really depends on which environment you are comfortable working with. You can also use Visual Basic or VB macros in Excel.

I myself would do everything in R, since it can perform all three tasks I mentioned. I routinely have to read csv files, do analysis and perform plots. Since usually I am working with multiple country data, I must produce a graph for each country. R lets me do this very easily. Furthermore R graphs are very customizable (see the graph on R project home page), and when you have a graph you can produce practically any format you like, see ?device.

Even in R you can achieve the aforementioned tasks in different ways. For example you can use only base packages, or use packages such as foreach, plyr, reshape for automation and data manipulation. For plotting you can either use base R graphics, lattice, or ggplot2.

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1 good answer to the general question. I may yet give R another try for this problem. I already find tapply very useful for creating summary statistics over groups in the data. $\endgroup$
    – Paul
    Commented Aug 20, 2011 at 5:14
0
$\begingroup$

I use matplotlib for the same purpose. This is my python code:

#!/home/user/miniconda2/bin/python
import sys
import csv
import numpy as np
import matplotlib.pyplot as plt
#plt.style.use('stylesheet')

if len(sys.argv) != 2:
    print "Single argument Expected"
    sys.exit(2)
f = str(sys.argv[1])

with open(f) as f1:
    reader = csv.reader(f1, delimiter=',')
    first_row = next(reader)

data = np.genfromtxt(f, delimiter=",", skip_header=1)
for col in range(1,len(first_row)):
    plt.plot(data[:,0],data[:,col],label=first_row[col],lw=2)

plt.title(f[:-4])
plt.xlabel(first_row[0])
#plt.ylabel(first_row[1])
plt.legend(loc=0)
#plt.show()
plt.savefig(f[:-4]+".pdf")
print "Done! Saved image to "+f[:-4]+".pdf"

Go through matplotlib documentation to know how to change plot style and other properties. I personally use a stylesheet.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.