3
$\begingroup$

As much of empirical research (or data analysis) has become a major software development exercise, practitioners use more and more such tools. So far I am aware of researchers using project management, issue tracking and versioning tools (JIRA, GitHub) but I think there should be a system to track and analyze (e.g. build a graph it, also for plotting) which data files a project uses, and which other pieces of code it calls. Is such a tool in common use for R or Stata?

I know that the major straight-up development tools for this are Maven, Ivy, and Gradle. But they do not seem to support R or Stata. What could one use for our use case? (This is why this question is on CV and not SO.)

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ For Stata I've used project from SSC for some time, and works wonderfully. For general purpose see GNU Make. One reference is oreilly.com/openbook/make3/book/index.csp. (I'm not sure this is on-topic here.) $\endgroup$
    – Roberto Ferrer
    Commented Jul 24, 2015 at 16:13
  • $\begingroup$ Why not just write a little wrapper to scan for ".dta" files in directories and subdirectories, and then scan for references in the ".do" files. Output this data to a txt file and archive it in the Github account. Then when you see the line-by-line, you can verify if things were created or destroyed, broken links, etc. $\endgroup$
    – AdamO
    Commented Jul 24, 2015 at 16:38
  • $\begingroup$ @AdamO That's great in theory. I think Gentzkow and Shapiro have a little Python script doing basically this. But I think they also wanted to start using a proper tool for this soon, and I just don't know which one they could mean. $\endgroup$
    – László
    Commented Jul 24, 2015 at 17:01

1 Answer 1

Reset to default
3
$\begingroup$

This isn't a fully automated system but Hans-Martin von Gaudecker at Bonn is an evangelist for doing this with waf, a Python-based build system that allows you to specify dependencies between files and only rebuilds what has changed since the last run or is downstream from such a change. It's pretty language agnostic. His templates support Python, R, Matlab and Stata out of the box and there's experimental support for Julia. As an added bonus, it automatically runs non-dependent tasks in parallel.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.