3
$\begingroup$

I'm involved in a census at our condo association. We may, for logistics reasons, survey 1/4 of the population per quarter so that we have surveyed everyone by the end of the year, but let's assume we survey everyone at once.

A year later, we survey again. Since we're attempting to cover 100% of the population each year, the results are what the results are each year, right? It wouldn't make sense to run last year's results (prior) and this year's results through Bayes Rule would it?

EDIT: In response to the excellent questions, I'm thinking of a single question on the survey that is asked both years. For example: what percentage of residents believe that the Management Office does a good job of handling complaints? At first glance, this seems textbook-like: you have a prior distribution based on the previous poll, you gather data, use Bayes, and now you have a posterior which in principle better reflects reality than the prior or current data. EDIT2: For this question, I'm assuming that there is no other data: we simply have (anonymous) polling results (and uncertainty) from one question asked in two different polls and wonder if the data for that question can be combined in some manner.

EDIT: As the answers point out, the problem is that the two samples are not independent, regardless of how far apart they are, because many residents will be asked both years. (Which gets me thinking: how independent is "independent"? Do we say that we need a method of choosing respondents that has less than 95% (99%, etc) probability of picking the same respondent twice?)

If we were taking samples from a much-larger population, I can see how Bayes Rule would be helping us to solidify our answers, but in this case... Actually, I'm not sure how using Bayes Rule on data that is evolving over time (polls, say) would work out, since the prior is "out of date" in some sense.

Questions:

  1. Does it make any sense in a census (attempting to survey the entire population) situation to run the prior census' results and the current census' results through Bayes Rule?

  2. If we only get a 50%, say, response rate would that change the answer to #1?

  3. Does it make sense to use, say, polling data as a prior for polling if we know that the data (opinions) are changing over time? (i.e. aren't priors a statement of prior belief about the current situation, not just arbitrary beliefs at some time in the past?)

EDIT: It appears that once the population's answers are subject to change, we then have to actually model this change (at least accounting for time), at which point we've moved beyond a simple application of Bayes Rule. IS that correct?

$\endgroup$
  • $\begingroup$ What do you mean by "running the results through Bayes rule"? What question are you trying to answer? $\endgroup$ – Aniko May 8 '12 at 13:42
  • $\begingroup$ @Aniko My guess is that it the scenario he uses one years results to get a prior uses the second years results for the likelihood and "runs the results through Bayes rule" to get a posterior distribution. But what inference problem is the posterior there for. The question is a bit too vague. I do think that if a complete census is obtained both times then you could look at how answers changed on individual and group responses. But there is no random sample and no inference problem. If data is complete in the first year but not in the second the information from the first could be used. $\endgroup$ – Michael Chernick May 8 '12 at 15:21
  • $\begingroup$ It would be used to impute. Then a Bayesian approach along the lines of what I surmised could be used to characterize the uncertainty in the second year sentence due to the imputation. This would also depend on the method of imputation. I think the simple rule of LOCF might not be the best as you may be able to borrow strength fromthose that responded in years 2. $\endgroup$ – Michael Chernick May 8 '12 at 15:27
  • $\begingroup$ @Aniko: I've edited my question, but when I wrote it I was thinking of a simple, textbook-like case: a single binary question on the survey. You have a prior survey's results and the new survey's results, yielding a posterior. Does that thinking make sense. (Several answers make it clear, it doesn't.) $\endgroup$ – Wayne May 8 '12 at 16:41
2
$\begingroup$

There are numerous examples of Bayesian analysis of census data in the book Bayesian Data Analysis by Gelman, Carlin, Rubin, and Stern, especially in chapters 5-8.

In fact, whenever the census data collection mechanism is not ignorable, this can often be a crucial part of the analysis. Consider, for example, prior beliefs about demographics that might be under-reported due to unusual household living situations. If you were targeting only a few specific covariates from the census data to estimate something about migrant workers, say, taking this into account would be extremely important.

That example might not be very realistic, but surely there are other examples with census data that highlight a similar point: Bayesian methods allow you to account for hierarchical aspects of data collection and to concentrate your assumptions into well-articulated priors. This would be important when seeking models for underrepresented demographics or non-ignorable designs.

More directly to your three questions:

(1) This seems too simplistic. I don't think you just "run the data through Bayes' rule". Bayesian analysis is the process of articulating a prior distribution (it could come from the past census, but probably you also know pieces of information that lead you to have a current prior belief that's somewhat different from just the most recent census (say, regarding job loss or something)), and articulating a likelihood model, and then using computational approaches to construct the posterior. You don't "run" the current census data "through" Bayes' theorem, unless that is your prior and you have already articulated some likelihood model for the problem you're trying to perform inference on.

I guess to answer (1) for you, we'd need to know more about what you're thinking of using as a prior, how you are splitting the data analysis out (are you setting up a hierarchical model, with hyperparameters sampled from some meta-prior, etc.?), what sorts of parameters / test statistics you are interested in estimating, and what your likelihood model is.

(2) You should probably be modeling things that effect the response rate.

(3) I think you are confused a little here. You seem to think that either last year's census or some polling data should be the "prior" and that this year's census should be some sort of check on the posterior. It's like you want to specify a model that converts old census data to new census data, but this doesn't make much sense to me.

I could be misunderstanding you, but it seems like you're thinking of Bayes' theorem as a black box that you drop data into... but it's certainly not that. You use Bayes' theorem to test your likelihood model of the data, incorporated with prior beliefs on the parameters of the model. Usually this is done in hierarchical stages to sequester different parts of the model from each other, generally because some subset of the data collection can be treated as ignorable conditioned on knowing certain hyper-parameters.

$\endgroup$
  • $\begingroup$ Yes, I'm definitely confused. I've edited my question to focus my confusion a bit, but I guess what I'm wrestling with (perhaps I'm so confused I'm wrestling in the stands instead of in the ring) is: a poll on a single question at two different times theoretically can be used, via Bayes Rule, to adjust what we believe the population as a whole thinks, right? But what about as the polling times grow farther apart (and hence the population is changing) or as the two polls become less independent (i.e. you're doing something like a census)? Not sure if that makes any sense. $\endgroup$ – Wayne May 8 '12 at 16:50
  • 1
    $\begingroup$ That makes sense, but then you should be modeling how correlated you expect them to be. Maybe you draw their correlation coefficient from a Beta distributions whose curvature is assumed to have a functional dependence on the time between the polling, or something. Then you run your analysis and get some posterior distribution over current opinion inferred from poll opinion. You check this against your data and see if it gives sensible results. If not, it could tell you that your model of census data is off, or your assumed correlation model is off. $\endgroup$ – ely May 8 '12 at 17:03
3
$\begingroup$

This is a bit hard to say without knowing what data you are trying to get. Political opinion? wealth?

This being said, the answer is, no, I do not think using the data from last year as a prior is appropiate. One of the main problems, is that you won't have independence, you will ask the same people again. Bayessian statistics does assume that you update your prior with new, indenpendent data. That should answer 1 and 2 as well, since with 50% polled each year you can still roughly estimate that half your populations is polled in both years.

Now to the third questions: Ignoring time is not a good idea as well, since it will lead to your model assumption being wrong if you do not include it in an appropiate form. This does not depend on the frequentist or bayesian framework

If you could track individuals over the census each year, you might be able to build some very nice models including individual variations and time, but this again depends on what you are trying to model/analyse. However, this is usually not possible due to data confidentially, at least when looking at an entire populations.

To conclude: It would be wrong to include the same person twice, unless you actually model that. Hierachical models can do so. This is true for both the Bayesian and the Frequentist approach. In both cases that means you shouldn't pool your data for an analysis, and in the Bayesian case the same person should not influence both the data and your prior. Again, unless you actually model individual persons.

It also wrong to pool a clearly time-dependent variable over different years, without modelling the influence if time. This is also true for both Bayesian and Frequentist data.

What you might do, with the appropiate discussion, is to choose a prior which is sort of loosely modelled after last year's data, if you start with last's years mean, estimate the largest plausible change in a year with expert knowledge and use that to build your prior. This will usually result in a prior similar in form to last year's posterior distribution, but with a much more wide shape. However, if you talk about surveying entire populations, your data should always dominate any reasonable choosen prior. Last's year posterior is not a reasonable choosen prior, because it will be too tightly focussed on a particular choice.

$\endgroup$
  • $\begingroup$ I was thinking of one exemplar question, such as "Do you approve of the way that the Management Office handles complaints?". It seems from textbook examples, that if you had two actual (independent, as you point out) polling results which were gathered closely in time (so that time can be ignored) you wouldn't need to model anything... Or maybe I'm just fuzzy and in fact you have to model your uncertainty otherwise you end up with two spikes for answers -- I'm thinking of modeling in the sense of a linear regression with multiple predictors, etc. $\endgroup$ – Wayne May 8 '12 at 16:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.