Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 7 down vote favorite
2

I am an R noob who is required to do various kinds of analysis on large data sets in R. So while looking around this site and elsewhere, it appeared to me that there are a lot of esoteric and less well known issues involved here - like which package to use when, what transformations to (not) apply on the data etc.

I am just wondering if there is a book/tutorial/guide that demystifies all this and presents the information in a systematic way? I prefer doing this instead of looking around, and collating information from different sources online.

Thanks in advance.

share|improve this question
2  
1  
It might help to share some examples of what kind of analysis you want to do and what your data looks like. Simple stats like means or complex regressions ? 200 variables accross a thousand rows, or 4 variables and 20 million rows ? – PaulHurleyuk Sep 9 '11 at 12:05
1  
If you have really "large" datasets, you should perhaps have a look at relational databases. A starting point for this can be the "R Data Import/Export" manual that ships with R. N.B. the manual is also available via the "Manuals" section of the R website. – user5644 Sep 9 '11 at 12:16
1  
First, important question: What do you mean by large, and what do you want to do? – EpiGrad Sep 10 '11 at 1:24

1 Answer

up vote 1 down vote

Here are a couple of blog posts I did on this subject of Large Data Sets with R. There are a couple of packages like ff and bigmemory that make use of file swapping and memory allocation. A couple of other packages make use of connectivity to databases such as sqldf, RMySQL, and RSQLite.

R References for Handling Big Data

Big Data Logistic Regression in R with ODBC

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.