Squeezing the juice from a large data set I may soon have temporary access to a large and interesting data set, where the data is sensitive and raises privacy and confidentiality concerns. Number of records in the low hundred thousands, number of variables in the mid-hundreds. I need to describe what I will do with the data, and provide assurances that I will not release anything that could identify any individual.
I have some particular questions that I want to answer, but boy, I sure wish I could take the data home and play with it for a year or three.
So I have been trying to figure out if there is some way to summarize the data that obscures all the observations and shrinks it a lot, but nonetheless lets me pose it new questions. 
The best thing I have been able so come up with is to take all the variables, and their squares and logs (and a couple of lags of each, if they'll let me at them) and then, what? Regress each on all the rest, and take home the coefficients? Take the first four or five own & cross moments? Run all the 3-variable regressions? (My background is in economics, so I think every statistical question is a regression).
I am mainly a policy guy, and more a consumer of statistics then a generator of new statistical knowledge. I have not found a literature on this question, and don't know if one exists. But it seems to me that the opportunity to bring home, not just answers to my current focal questions, but also a summary rich enough that I could hand it to a colleague with a different question and s/he could answer it from the summary, is an opportunity I should seize if at all possible. 
 A: Since you haven't described the exact fields which are privacy sensitive (which will be removed before you're allowed to "take home" the data), I'm assuming these would be fields such as names, addresses, phone nos, income or expenditure details, employers, etc.
If you'd prefer to fit a regression on interesting fields, then I would suggest storing the relative importance of independent variables is more useful than storing the coefficients as-is.
I suggest you could try these steps to generate additional "features" from this data:


*

*Use last names to find concentration of demographic groups within the population, although keeping in mind it doesn't lead to your work being used for racial profiling (we all have to be sensitive about these things when working on data).

*Use addresses to get the approximate geospatial location of each individual, and generate features indicating concentration of these. You could first perform suitable clustering to get members and center points of clusters, and then round off the geospatial coordinates so that they are at a more coarse level sufficient to only identify an individual's city/town/county and not their exact home location. If location is an important consideration, the other details such as climate of the location and topography may also be relevant details to include.

*If income/expenditure is part of the dataset, then generate parameters for them such as identifying most suitable probability distribution an income fits, then mapping each records to get how many std. dev. it is away form mean (if normally distributed), etc. Also cluster it and add cluster centers and distance to center.

*If employers are listed, then determine the nature of employer such as govt. sector, industry type, size, geographic location. All these details about a firm are available publicly.

*If dates are part of the dataset, then you could identify the season and part of financial year, etc. from the date and retain those as new features.

*If the data is time series, then you could determine auto-correlation, or, perform some causality tests based on your hypothesis.


You can think along these lines and check with the owners of the data to find out which features they would permit you to take away.
A: If you have all categorical data then you can make a table like this
var1 var2 var3 N
0    0    0     2 
1    0    0    10
1    1    0    15
1    1    1     2
0    1    0     0
0    1    1    10
0    0    1     7

This will allow you to exactly reconstruct the data for those three variables. You probably won't be able to make such a table for all variables in your dataset and you may have to coarsen some of the categories so you won't end up with too small frequencies which would violate confidentiality.
If you have all continuous data and are interested in linear models then a table of all the variances and bivariate covariances plus a table of the means is enough to estimate any linear model, including structural equation models. 
