23
$\begingroup$

I am looking for input on how others organize their R code and output.

My current practice is to write code in blocks in a text file as such:

#=================================================
# 19 May 2011
date()
# Correlation analysis of variables in sed summary
load("/media/working/working_files/R_working/sed_OM_survey.RData")
# correlation between estimated surface and mean perc.OM in epi samples
cor.test(survey$mean.perc.OM[survey$Depth == "epi"], 
    survey$est.surf.OM[survey$Depth   == "epi"]))
#==================================================

I then paste the output into another text file, usually with some annotation.

The problems with this method are:

  1. The code and the output are not explicitly linked other than by date.
  2. The code and output are organized chronologically and thus can be hard to search.

I have considered making one Sweave document with everything since I could then make a table of contents but this seems like it may be more hassle than the benefits it would provide.

Let me know of any effective routines you have for organizing your R code and output that would allow for efficient searching and editing the analysis.

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Just to avoid copy/paste, sink() or capture.output() might be your friends. Reporting utilities, like Hmisc, Sweave, or brew are worth to consider (your point 1). Versioning systems (rcs, svn, or git) might help with point 2. $\endgroup$
    – chl
    May 19, 2011 at 21:53
  • $\begingroup$ @chl - thanks for the suggestions. I wasn't aware of sink() and capture.output(). Thats great. $\endgroup$
    – DQdlM
    May 20, 2011 at 13:15
  • 1
    $\begingroup$ today there is also knitr! $\endgroup$
    – kjetil b halvorsen ♦
    Jul 4, 2017 at 22:13

4 Answers 4

Reset to default
23
$\begingroup$

You are not the first person to ask this question.

Improve this answer
$\endgroup$
4
  • $\begingroup$ +1 and the first link you provide references a thread therein :-) $\endgroup$
    – chl
    May 19, 2011 at 21:45
  • $\begingroup$ @chl Thanks! I was wondering if this question is a duplicate and should be closed... $\endgroup$
    – Bernd Weiss
    May 19, 2011 at 21:54
  • $\begingroup$ It is, IMO. But as there are no votes to close, I'm reluctant to close it. Also, the older one was more general, but a very similar question has been closed in the past. Let's wait and see how it goes. $\endgroup$
    – chl
    May 19, 2011 at 21:58
  • $\begingroup$ thanks for the list! This is very useful. I figured I wasn't the first to have this question but I didn't seem to find much with my (obviously inept) initial search. $\endgroup$
    – DQdlM
    May 20, 2011 at 13:01
6
$\begingroup$

I for one organize everything into 4 files for every project or analysis. (1) 'code' Where I store text files of R functions. (2) 'sql' Where I keep the queries used to gather my data. (3) 'dat' Where I keep copies (usually csv) of my raw and processed data. (4) 'rpt' Where I store the reports I've distributed.

ALL of my files are named using very verbose names such as 'analysis_of_network_abc_for_research_on_modified_buffer_19May2011'

I also write detailed documentation up front where I organize the hypothesis, any assumptions, inclusion and exclusion criteria, and steps I intend to take to reach my deliverable. All of this is invaluable for repeatable research and makes my annual goal setting process easier.

Improve this answer
$\endgroup$
0
2
$\begingroup$

Now that I've made the switch to Sweave I never want to go back. Especially if you have plots as output, it's so much easier to keep track of the code used to create each plot. It also makes it much easier to correct one minor thing at the beginning and have it ripple through the output without having to rerun anything manually.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Sweave is wonderful. It takes some getting used to, but if you already know TeX and R, its the obvious choice. It also allows you to never spend time aligning table columns ever again, which is nice. $\endgroup$
    – richiemorrisroe
    May 20, 2011 at 9:22
  • $\begingroup$ thanks for the input. My 2 concerns with Sweave are 1) I will wind up with a bazillion files in my directory - especially with a lot of figures, and 2) I will have to be really careful with the code to prevent hiccups each time I compile the whole doc (e.g., I think something is loaded and its not). Do you have these problems? $\endgroup$
    – DQdlM
    May 20, 2011 at 13:06
  • $\begingroup$ 1) You can choose to delete the intermediate files if you want; I use the Sweave.sh script (cran.r-project.org/contrib/extra/scripts/Sweave.sh) which does so automatically; though can easily be turned off. If you do that, make sure you know what it will delete before using it to prevent possible disaster. The short version is that if no files share the basename of your Rnw file you're okay. $\endgroup$
    – Aaron left Stack Overflow
    May 20, 2011 at 13:13
  • $\begingroup$ 2) In my opinion, having to be careful in that way is a Good Thing, and will sometimes compile my Sweave document with that exact purpose in mind, that is, to make sure I've properly kept track of everything needed to recreate the analysis. $\endgroup$
    – Aaron left Stack Overflow
    May 20, 2011 at 13:16
  • 1
    $\begingroup$ @naught101: For long analyses, I do run it separately and save the results, usually in an .RData file, for input by the Sweave document. However, there are also several great options out there for "caching" the results from a code chunk so it doesn't get rerun. $\endgroup$
    – Aaron left Stack Overflow
    Jul 20, 2012 at 12:16
2
$\begingroup$

For structuring single .R code files, you can also use strcode, a RStudio add-in I created to insert code separators (with optional titles) and based on them - obtain summaries of code files. I explain the usage of it in more detail in this blog post.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.