Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite
1

This is, unfortunately, something I was never exposed to as an undergraduate or in my PhD core (economics).

As an example, suppose I'm looking at three well-known nationally representative surveys: recent estimates from the American Communities Survey (ACS), the General Social Survey (GSS), and the NLSY79. Suppose each has an aggregate variable I'd like to use in an analysis of income, and further suppose each of the three independent variables are unique to their respective surveys.

I suppose cross-sectional analysis is out, since I'm dealing with different respondents. It also seems like doing a time-series analysis would be tricky if the surveys were administered at different times (the examples are).

In short, what can I do, if anything, to observe interesting patterns in the data? What's kosher, and what's faux pas with regard to using results from different surveys?

share|improve this question
2  
Although this is not a meta-analysis as such, there is much to be learned from that practice in terms of evaluating the quality and comparability of separate studies. – whuber♦ Nov 3 '11 at 4:54
3  
Andrew Gelman has a paper that uses surveys from Gallup and GSS simultaneously to examine state level effects of death penalty opinion. Perhaps that would give some useful references to this problem. – Andy W Nov 3 '11 at 12:35
why do you want to do this? Is there a particular subgroup you want to analyze that is too small? – Jonathan Nov 3 '11 at 20:16
@Jonathan, it isn't about subgroups being small, it's that different surveys collect different variables. For example, the GSS and the NLSY97 have personality variables, while the Census and NLSY79 do not. – jrhorn424 Nov 4 '11 at 3:09
1  
@jrhorn424 I may be approaching this from a totally different viewpoint... my past experience with combining survey data is primarily to achieve a larger sample or to combine separate populations/time periods that were separately surveyed. Are you actually planning to combine the surveys at the data level? With distinct variables in each data source it sounds more like you would separately cite the surveys when analyzing separate topics, and it would be combined more at the "insights" level. – Jonathan Nov 4 '11 at 20:32
show 1 more comment

1 Answer

up vote 2 down vote accepted

what can I do, if anything, to observe interesting patterns in the data?

Generate hypotheses and areas of interest by talking with other economists, policy makers, subject matter experts, whoever you think your audience is. As you form these ideas, attempt to confirm or disprove hypotheses. Don't create too many abstractions at first (variables aggregated from multiple variables, simplifications of variables, etc.) because you are still exploring. As you investigate your ideas and hypotheses, you may come up with some new ideas.

what's faux pas with regard to using results from different surveys?

Combining data from multiple surveys is very risky unless the surveys were actually designed to be combined, as in wave surveys. There needs to be a good reason to combine the data. Faux pas:

  • Combining surveys with incompatible sampling procedures
  • Ignoring the potential effects of different field times, different survey formats, etc.
  • Combining data from questions that were framed differently, had different responses available, etc.
share|improve this answer
Thanks, @Jonathan. The first answer isn't really what I was looking for... perhaps I need to rephrase the question. The list of faux pas was quite useful, however! – jrhorn424 Nov 4 '11 at 3:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.