We did a quota sample going for age group, sex, and education level. Given that we asked strangers on the street, we could only assess education level after speaking to the people. Given the short questionnaire, we ask the questions anyway.
Now we have about 500 filled in questionnaires, but can match the quota only with about 250 of them. If we were to use all participants, academics would be over-sampled (university towns), while non-academics would be under-sampled (both men and women, more severely for women). To analyze the data with 250 cases, we take everyone from the limiting combinations. But what do we do with the other combinations which are over-sampled?
- Just do a random drawing in these over-sampled combinations (so the number of cases matches the quota for that combination, e.g., we have too many academics in some age groups from which we can draw randomly)?
- Draw repeatedly from the over-sampled combinations, and average the results?
- Depends, depending on ...?
- Some other way?
Not sure how to proceed here.
Edit: I forgot to mention, we aim for a representative sample (using distributions of the age, sex, and education level in the population to determine the quotas).