0
$\begingroup$

I conducted a web-based survey. Started with 1000+ respondents and it gradually reduced to 400+. Some key questions were answered, for example key question re dependent variable is answered by N=750. However, demographic questions were at the end and not completed by all. Question:

1) can I use the questions (especially for descriptives) that have been answered but that I may not have a complete survey off

2) how do I present my research, N=what?

Missing data imputation is not an option because missing data are not at random. Thanks!

$\endgroup$
1
$\begingroup$

It's good that you placed the demographic questions at the end of your survey. This is an example of good survey design and demographic questions -- especially the household income question -- tend to be more sensitive and survey participants are frequently less likely to complete them than less invasive questions. This is why they are usually placed at the end.

To answer your questions, yes, you can use only the partially completed data, but if you do so, you should be clear in your presentation what how many people answered the questions you are reporting on (i.e. what the denominator is) as this value changes as participants in your survey dropped off. You should provide an overall completion rate (how many answered all question) and then for sub-analyses, indicate how many completed that question or questions for bivariate analyses. The American Association for Public Opinion Research has more detailed explanations of how to report response rates and provides a calculator to help you compute them: http://www.aapor.org/AAPORKentico/Education-Resources/For-Researchers/Poll-Survey-FAQ/Response-Rates-An-Overview.aspx

Most people will be concerned about the generalizability of your findings if a lot of people refused to complete the substantive portions of your survey. The demographic questions can often be used as a gauge to the generalizability of your findings (e.g. if you have some people completing the demographics but not certain questions, you can use the demographic data to compare to those who completed the survey in it's entirety to make a judgment call about whether or not those who completed all questions are in some way different from those who only partially completed questions).

If you can obtain data on the characteristics (perhaps in your sampling frame) about those who abandoned your survey, it would be recommended to compare those characteristics to those who completed. If there differences are minimal, you would have increased confidence in the generaliability of your findings.

$\endgroup$
  • $\begingroup$ Please mark this answer correct if you've found it satisfactory. $\endgroup$ – StatsStudent Jan 8 '16 at 5:17
1
$\begingroup$

It is a common misconception that it is preferable to conduct an analysis using only the complete cases, because the assumptions required by various imputation methods may not hold. A complete case analysis makes the extremely strong assumption that missing data are missing completely at random (MCAR). MCAR is very rarely satisfied, but e.g. when there is hardly any missing data it may not make a practical difference exactly what one assumes about missing data. If there is a substantial proportion of missing data MCAR is in most applications an inappropriate assumption, in which case analyses that assume MCAR (such as a complete case analysis) are inappropriate.

Many missing data imputation methods (e.g. forms of multiple imputation or direct likelihood methods) assume that missing data are missing at random (MAR) and it was not clear from the original post whether the original poster was concerned that MCAR or MAR would not hold. MAR at least allows missingness to depend on observed data. If the missing data are really missing not at random (MNAR = anything that does not fall under MAR), then there are still imputation approaches that can handle this, they just require assumptions about the missingness process and one can investigate the impact of a range of different assumptions. One or more reasonable attempts at imputation are preferable to conducting an analysis that is only valid under MCAR.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.