My university is running an anonymous survey, mostly to check if we understand how we are going to be assessed, if we are comfortable with the material, and if we find the material well organised. Participation was low, so, one of the lecturers said they will withhold our assessment grades until participation reaches at least 2/3 of the group. This made me think if that could bias the answers, as the motivation is getting their mark rather than a genuine participation to provide feedback?
8
-
1$\begingroup$ Seems to be a bit of a rock vs. hard place scenario. If there was no "coercion", then the subjects would all be answering voluntarily, and that woudl be a biased sample (probably mostly students who have strong opinions about the class, strong enough to bother answering the survey). If on the other hand, the survey was mandatory, then it would not even be a sample anymore (but the all student population for the class). Would a student answer differently, as a revenge to being forced to answer the survey? Possibly, but not too likely. ...ctd $\endgroup$– jginestetCommented Nov 18 at 20:50
-
1$\begingroup$ ...ctd. And if participation reaches 2/3 of the population, it is still biased (because sampling is not random). So which of these biases is less? Usually, a low participation is quite biased (as explained in the earlier comment). If participation reaches 2/3, it is less likely to be so, as your sample size nears the population size (so even that 1 unlikely spiteful answer weighs less). But, in general, any non-random sampling will be biased, in some way... $\endgroup$– jginestetCommented Nov 18 at 20:53
-
$\begingroup$ Is the survey anonymous or not? $\endgroup$– whuber ♦Commented Nov 18 at 22:34
-
$\begingroup$ @whuber Yes, it is anonymous. $\endgroup$– PorterCommented Nov 19 at 7:09
-
1$\begingroup$ See Satisficing. People may simply rush through the survey as an answering strategy. As a side note, even if the university tries to force answers, they may still get no response from people who simply dropped out or who have other personal problems. So trying to force it is not a guarantee that non-response will drop to 0%. I'd suggest to identify why there's such a non-response rate. Nonresponse in student surveys is a subject that has been quite discussed in survey methodology literature. $\endgroup$– J-J-JCommented Nov 19 at 7:10
|
Show 3 more comments
1 Answer
$\begingroup$
$\endgroup$
Satisficing is at least one problem to consider. For instance, people may simply rush through the survey as an answering strategy, in particular if the survey is long.
Non-response bias may be something to consider too: Will the people who respond to such an incentive have a different profile from those who do not? For instance, simply consider the case of people who dropped out from the university.
It won't necessarily bias the results that bad depending on the context, but it could. Identifying the reasons for non-response may be useful in this regard.
Generally speaking, unit non-response in student surveys is a subject that has been discussed in survey methodology literature, as a search in your favorite academic search engine will certainly show. In recent papers, Steinmetz et al. (2020) provide a very interesting discussion about increasing response rates (the context is international student surveys conducted in Sydney, Australia). See in particular pp.1114-1117 of their paper.
There could be also things specific to your university explaining the current non-response rate. Hard to tell without more information and details. Anyway, this might involve questions about university management and governance, that could go beyond the scope of statistics or of a Q&A website.
You may find that even proponents of mandatory student surveys suggest to exercise caution, see for example pp.67-68 of Fass-Holmes (2022).
References
Steinmetz, Christine, Sian Thompson, and Nancy Marshall. "Surveying international university students: The case of the 5% response rate." Issues in Educational Research 30.3 (2020): 1105-1125. http://www.iier.org.au/iier30/steinmetz.pdf
Fass-Holmes, Barry. "Survey Fatigue--What Is Its Role in Undergraduates' Survey Participation and Response Rates?." Journal of interdisciplinary Studies in Education 11.1 (2022): 56-73. https://files.eric.ed.gov/fulltext/EJ1344904.pdf
