3
$\begingroup$

I'm looking to do some research with the GSS (the General Social Survey; a survey that asks over a 1000 people every year various questions and collects their demographic information as well). I'd like to look at one of their questions and see how predictors to that question change over time. The response variable that I will be looking at is a 4 level ordinal variable (strongly against to strongly in favor). I will be using a cumulative logit model with demographic variables to predict the response. For example, does income predict where the respondent answers on the likert scale.

While this could be interesting, I think it would be more interesting to see how these demographic predictors change over time. For example, does income predict income in 1972 but not in 2000? (Just to clarify, it's a different sample of people every year and the weights change per year if I recall correctly).

My issue is how I would approach this.

  1. Would I just run a separate regression for every year?

  2. Alternatively, would I include the year as an interaction term like so?

    respondent opinion = income * year + age * year + ...
    (with the possibility for other interaction terms)

  3. Would it make sense to just collapse the variable into mostly agree and mostly agree? I would sacrifice granularity, but interpretation could be easier?

I'm not familiar with any time dependent data analysis and was told by my professors to stay away from it until I've taken a class on it. If any of these options are wrong, could you explain why (basic theoretical foundation) so I know what I'm doing wrong.

$\endgroup$
1
$\begingroup$

What you are trying to do, using data from multiple years collected on a different sample each year, is commonly known as pooled cross-sectional analysis. The basic considerations for doing pooled cross-sectional analysis on continuous outcomes also generally apply to generalized linear models, including the ordered logit model you described above.

  1. If you pool your datasets to run the regression, you will need to "rescale" (adjust) your weights. How you rescale your weights depends on what kind of weights they are (e.g., whether it is a sampling/population weight based on study design or something else). The documentation that comes with your data will tell you what kind of weights they are. For example, in the Demographic and Health Surveys that I often work with, the samples are meant to be nationally representative, so the sampling weights correct for the probability of selection into the sample. One of the DHS statistical experts, Ruilin Ren, for example, suggests dividing the weight by the ratio of observations in the sample for the survey to observations in the population (Ref: DHS User Forum).

  2. Related to #1, if you are controlling for sampling design in your estimation, then you will need to ensure that the clusters and strata are specific to each survey. This is important because clusters change over time, especially if administrative boundaries are not stable (think census block groups). Very often, the same cluster number will be used from year to year, and if you use the clusters as-is, your statistical package may think they are all from the same cluster, irrespective of the time component, and incorrectly adjust your standard errors that way. A good strategy is to tack on the survey year at the end of the cluster. For example, if you have 10 clusters, change the cluster #10 from 1972 to 10_72, 1973 to 10_73, ..., 2000 to 10_00, and so on for each of your 10 clusters.

  3. There's a number of ways you can go about doing this estimation. Using survey year interactions with your predictors of interest is certainly one of those ways. An advantage of using the interactions is that you can test for structural changes over time using the Chow test (See Wooldridge, 2003).

  4. If you are comfortable running and interpreting an ordered logit model, I do not see why you should collapse your responses into a binary variable.

  5. Since you are using data from multiple years, you will also need to deflate income using a price index, such as the Consumer Price Index (CPI) to ensure comparability of income across all years.

Reference

Wooldridge JM. Introductory Econometrics: A Modern Approach. Thomson South-Western; 2003.

$\endgroup$
3
  • $\begingroup$ I'm not entirely clear on your second point, mainly because I don't have a strong understanding of experimental design. Could you point me to relevant sources so I can better understand what you are recommending? $\endgroup$
    – macworthy
    Jul 6 '15 at 1:09
  • $\begingroup$ 1. The Wooldridge reference I mentioned above is a good place to start and a worthwhile investment if you are interested in learning basic and some advanced analytic methods. A newer edition of the book is available on google books (link: books.google.com/…, Chow Test for pooled cross-sectional analysis starts on p.449) $\endgroup$ Jul 6 '15 at 1:42
  • $\begingroup$ 2. Since the data you're working with are survey data, I suggest reading up on analyzing survey data. One of my favorite resources for statistical computing is IDRE at UCLA, which has an intro to survey analysis self-guided seminar: ats.ucla.edu/stat/seminars/svy_intro. Be sure to also read the documentation that comes with your datasets. $\endgroup$ Jul 6 '15 at 1:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.