Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

EDIT: The Web Technologies and Services CRAN task view contains a much more comprehensive list of data sources and APIs available in R. You can submit a pull request on github if you wish to add a package to the task view.


I'm making a list of the various data feeds that are already hooked into R or that are easy to setup. Here's my initial list of packages, and I was wondering what else I'm missing.

I'm trying to limit this list to "real time" or "close to real time" data feeds/APIs, where the underlying data might change between downloads. There's plenty of lists out there for static datasets, which only require one download.

This list is currently biased towards financial/time series data, and I could use some help expanding it into other domains.

Free Data:
Data Source - Package
Google Finance historical data - quantmod
Google Finance balance sheets - quantmod
Yahoo Finance historical data - quantmod
Yahoo Finance historical data - tseries
Yahoo Finance current options chain - quantmod
Yahoo Finance historical analyst estimates - fImport
Yahoo Finance current key stats - fImport - seems to be broken
OANDA historic exchange rates/metal prices - quantmod
FRED historic macroeconomic indicators - quantmod
World Bank historic macroeconomic indicators - WDI
Google Trends historic search volume data - RGoogleTrends
Google Docs - RGoogleDocs
Google Storage - RGoogleStorage
Twitter - twitteR
Zillow - Zillow
New York Times - RNYTimes
US Census 2000 - UScensus2000
infochimps - infochimps
datamarket - rdatamarket - requires free account
Factual.com - factualR
Geocode addresses - RDSTK
Map coordinates to political boundaries - RDSTK
Weather Underground - Roll your own
Google News - Roll your own
Earth Sciences netCDF Data - Roll your own
Climate Data - Roll your own
Public health data - Roll your own
OAI Harvester - Open Archives Initiative harvester
RAmazonS3 - S3 Amazon storage server
Rflikr - Flikr api

Requires a subscription:
Bloomberg - RBloomberg
LIM - LIM
Trades and Quotes from NYSE - RTAQ
Interactive Brokers - IBrokers

share|improve this question
1  
depends on what you mean by 'easy to set up'... I have a number of custom scripts that are 'easy to set up' –  David Jul 5 '11 at 19:44
2  
Right now I'm defining "easy to setup" as "the package exists" or "someone's posted/published a function to download the data" –  Zach Jul 5 '11 at 20:07
    
I would further define "the packages exists" as "the package is on CRAN" or "the package is on R Forge," "the package is on some CRAN-like, public repository." –  Zach Nov 10 '11 at 17:05
    
Addition: Federal Register API (in the works). –  Fr. Dec 2 '12 at 10:54
    
This thread may also be of interest: Import stock price from Yahoo Finance into R. –  gung Dec 19 '12 at 2:42

3 Answers 3

up vote 7 down vote accepted
  1. Instructions for using R to download netCDF data can be found here, a common format used for storing Earth science data, e.g. as in marine geospatial data from OpenEarth or climate model driver and forecasts from UCAR

  2. rnpn (under development) enables you to get data from the National Phenology Network - a citizen science project to track the timing of plant green-up, flowering, and senescence. See the developer's blog post.

  3. -obsolete- RClimate provides tools to download and manipulate flat-file climate data (with tutorials, including here-

  4. Download historical finance data with tseries::get.hist.quote

  5. Michael Samuel's documents downloading public health data

  6. raster::getData provides access to climate variables via worldclim

share|improve this answer
    
What is "netCDF data?" –  Zach Jul 6 '11 at 13:16
    
@Zach NetCDF is a kind of universal structured way of storing large datasets. An alternative approach relies on the HDF5 format. –  chl Jul 6 '11 at 19:58
    
@David (+1) Many thanks for the fourth point! –  chl Jul 6 '11 at 19:59
    
What happend to RClimate? I cannot find the package anywhere –  Daniel Ryback Mar 16 at 20:20

There's a project aimed at creating R packages with this objective (R interface to real-time APIs) called rOpenSci, which has 18 packages currently available or in development. Some (rnpn, rfishbase) are on your list already.

Great list! and full disclosure - I'm part of the rOpenSci project.

share|improve this answer
  1. ONETr - efficient interaction with the O*NET™ API, offering occupational descriptor data from the U.S. Department of Labor.
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.