8
$\begingroup$

In performing exploratory data analysis, I will often print out the graphs and write out comments/annotations etc.

Do people have suggestions for a better electronic methodology? I am especially interested in python/R.

I am looking for something 'quick (and dirty)' that doesn't slow down the exploratory work, but helps logging insights I have made.

What I could imagine is generating graphs as PDFs and then adding comments.

Ideally I would like the option to do this programmatically, so that if I redo the graphs I can 'automatically' add the comments back in.

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ What I sometimes do, in R, is to add a separate plot (on a separate page at the end of the pdf) and use that to paste some comments. $\endgroup$
    – hplieninger
    Commented Jan 25, 2017 at 16:08
  • 3
    $\begingroup$ This looks more like a programming question and if I am correct you might get better responses on R-help or StackOverflow $\endgroup$
    – mdewey
    Commented Jan 25, 2017 at 16:53
  • 1
    $\begingroup$ @mdewey, well I view it as statistical workflow in R - I agree my 'suggested solution' is just 'how to write comments to a pdf', but I am assuming that statisticians have faced this problem before and might suggest a totally different approach to the general problem of keeping track of notes associated with graphs. $\endgroup$
    – seanv507
    Commented Jan 25, 2017 at 17:00
  • 3
    $\begingroup$ My students do this without prompting by pasting graphs into MS Word and adding comments. I appreciate the interest in all-singing, all-dancing automated methods, but some times low tech is the best tech. $\endgroup$
    – Nick Cox
    Commented Jan 25, 2017 at 19:13
  • 1
    $\begingroup$ I think this is of general interest. I've taken the liberty of watering down the emphasis on python/R, which is contrary to guidelines here. $\endgroup$
    – Nick Cox
    Commented Jan 26, 2017 at 13:05

5 Answers 5

Reset to default
5
$\begingroup$

Here's an easy solution that many people have found useful. If you find it trivial, I won't disagree. This cuts across statistical software, operating system and other computing details.

Just copy and paste your graphs into your favourite word or text processor and then add your own comments. That could mean MS Word, software supporting TeX, LaTeX, etc.

That's it. Clearly the advantages are simplicity (nothing new to learn) and flexibility (add what you want in the way that you want it).

This isn't an automated solution. But even automated solutions depend on being fed information on the graphs and your comments, so what is that different?

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ I think you are underselling it. If rather than 'pasting' you add a link to a file ( as you can in word latex etc) then as graphs are updated the document updates - so 'automated'. Additionally, I like the fact that one can separate the generation of the graphs from the layout ( eg fit 3 graphs on a page vs 2 x 2 layout) $\endgroup$
    – seanv507
    Commented Jan 26, 2017 at 17:47
3
$\begingroup$

I highly recommend Jupyter Notebook, which lets you create documents that contain interspersed code blocks, plots, and notes/documentation. The document can include markdown and latex, which is automatically rendered (much like writing on CrossValidated). When you run a code block, any text output and plots that it generates are added inline to the document. You can change a code block and re-run to update the output/plots. This is nice for testing things interactively (e.g. tweaking code/parameters to see what happens). I think it's easier than having to export figures and and paste them into a traditional, static document, especially if you change anything. You can export a notebook to PDF, etc. to get a static copy.

It's open source and works with Python, R, and other languages. The interface is browser-based, so it's cross-platform and easy to share notebooks. You can run the backend on your own machine, or you can host notebooks on a website so you/others can edit/view/run them from anywhere (the code will run on the server). Apparently there's a way to configure the notebook as the frontend to a compute cluster for parallel computations.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I have tried that on numerous occasions, but am not really convinced that it worked very well: Combining code /graphics/ documentation whilst ensuring everything is readable seems very hard in practise. I have seen presentations where this has been done but suspected the effort involved is considerable (as for any presentation). My point about EDA is that you are doing a lot of graphs etc - too many to present... in a presentation you show the best. Would welcome examples contradicting my impression. $\endgroup$
    – seanv507
    Commented Feb 19, 2017 at 21:28
2
$\begingroup$

I tend to do more and more analyses in R notebooks within RStudio. This way, I can have code, annotations and graphs toghether in one place and don't have to produce pdfs all the time - which is a real time saver. You write text and code in an editor and by clicking on a button, the code is executed (and the graph drawn) in place. So text, code and plot stay neatly together. It is also very easy to convert into HTML or PDF by some mouse clicks. I don't know, how well this works with Python, as I mostly use R.

Improve this answer
$\endgroup$
2
$\begingroup$

It sounds like you want some kind of literate programming. R affords Sweave, and Knitr that interface with LaTeX. There are other options for different kinds of output formats, such as ODFweave for editable documents (like Word documents), and RMarkdown that can output multiple types (like HTML in addition to the above mentioned). Other statistical software commonly have analogous features.

(There is a bit of up-front work in using these. I typically do one-off projects instead of longer-term projects that require regular, and similar, reports, so I typically use @NickCox's method of dumping or copying to a file and writing comments around it.)

Improve this answer
$\endgroup$
1
$\begingroup$

In R: Sometimes I add an extra plot to a pdf with some basic information. This is most useful if the annotations are short and relate to the plot data such that you can paste extra information. For example:

pdf("cars-plots.pdf")
plot(cars)
plot.new()
legend("center", bty = "n", legend =
           paste0("Data: 'cars'\n",
                  "cor = ", round(cor(cars)[1, 2], 2), "\n",
                  "N = ", nrow(cars), "\n",
                  Sys.Date()))
dev.off()

Alternatively, if you have longer annotations, producing a report using R Markdown might be a solution.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.