The stat packages which I've been using are not very well suited for working with large data sets, i.e. more than 1 billion records. They all tend to try loading all into memory, and install one one machine.
Is there a statistical package that's comparable in uesability to Stata or MATLAB that can distribute work across multiple nodes (machines) and run regressions (panels, longitudinal) on large data sets? The idea's that it's got to be an interactive environment, but able to distribute processing when necessary.