Project management for remote collaboration in prediction Are there any tools for remote collaboration in prediction or machine learning settings? 
I am looking for a computing environment that includes appropriate source control, keeps track of how different datasets are being matched to different algorithms, and facilitates blending of various predictions.  What tools would be useful for this?
 A: I don't know of any off-the-shelf products specifically designed for collaboratively building predictive models, but I do think you can roll your own solution starting from a good version control system like git or hg and scripting tasks to make every step reproducible.


*

*All datasets should absolutely stay out of version control, but you should write shell or SQL or python etc. scripts that fetch the raw data from your various sources, and perform any data "munging" type tasks (filtering, cleaning, transforming, and so on).  These data manipulation scripts should be tracked in version control, and I like to name them such that if there are dependencies or an implied order, this is maintained in the natural (alphabetic) directory listing.  For instance I may have scripts:
01-fetch_census_data.sh
02-scrub_census_incomes.sh
03-60train_20test_20val_split.sh
03-75train_25test_split.sh
...
and so on.  Depending on how much time you want to invest, you can write code to cache intermediate results so long running steps are only run when necessary, or have dependencies automatically identified and handled.  

*Similarly you'll have some amount of code written in your favorite model development language that will take in processed datasets and perhaps learned model (hyper-)parameters and will produce some output in the form of learned model parameters, or predictions on some dataset.  You want to track this code, but you also want to track in version control the learned parameter values (for later blending).

*Finally you'll need some top-level driver script that picks out the appropriate sequential combination of data processing and modeling algorithm scripts.  This should represent a complete end-to-end experiment starting from the raw data and ending with a trained model (and likely some form of evaluation ex. plots or performance metrics which again can be kept under version control for reference).

*The important thing to keep in mind is that any of your collaborators should be able to simply take a clone of your repository and assuming they have the necessary access rights to your raw data sources, completely reproduce (and extend) any of your experiments

*If you use R, check out something like the ProjectTemplate package as a good skeletal starting point for your project

A: These might be helpful: Kepler, Process Makna, Taverna, and this paper.
