Reliability of online surveys

Question

I'm trying to get an idea about reliability of online surveys: I found some indication that "internet-based surveys produce data that is at least as reliable, valid, and of equal quality as data obtained more than via paper-and- pencil survey method", but I was wondering how to assess a survey's reliability and which issue could online surveys have.

For example, I run into a survey (in Italian) about special classes (in which disabled students are separated from normal students), that the author used to conclude how many students, parents, teachers and other school's figures agree with those classes; but the sample was self-selected, so there was no control on who voted voted, if they really were involved in school, and if they voted where they should have (there was a question divided by role, so four identical questions "Reply only if you are a student/teacher/parent/other", with no control on who really voted).

Also, in the article there is no mention on how they addressed those issues, or if they even tried something in that regard, and they said they just did an automated tally.

I was thus wondering how reliable could such a survey be, and in general how reliable are surveys with self-selected samples and/or questions to be answered only by a part of the sample (but with no control on who actually answers), and if there is some indicator to be aware of that could indicate a bad designed or implemented online survey.

Answer 1

The quoted statement in your question seems to me preposterously broad. Both internet and paper questionnaires vary hugely in reliability and validity. Some of the issues are the same in both formats. E.g badly written questions, badly written answers (if it's multiple choice), inadequate coverage of a topic, sensitive questions, etc.

In addition, internet based questionnaires have some additional problems. As J-J-J pointed out in a comment, some (including the one you linked to) can be filled out multiple times by the same person.

The random sampling problem is huge in both internet and paper (as well as phone and other) methods of administration. And it can be huge. However, if you mail or phone a survey, you have at least a little knowledge of who filled it out. It's probably someone at the address or phone number you gave.

For sensitive questions, there are additional problems, some of which may be easier to address over the internet. E.g. you can implement random choice or list selection more easily via computer. I haven't looked at the research in a long time, but, at one point, there was data showing that different ethnic groups (in the US) reacted differently, with some answering more honestly over the net and some more honestly in person. OTOH, methods such as ACASI and ACATI would be hard over the internet (although maybe there is now technology to do that, like I said, it's been a while since I looked at the research).

You can do a good survey online or in person or by mail. And you can do bad ones all three ways, as well. All of them take more effort to do well than most people think. Internet ones may take even more care than paper.

Answer 2

In addition to Peter Flom's answer (+1), a problem affecting specifically the reliability of online surveys is that they require an Internet access and the capability to use it.

So you have to make sure your population of interest not only want but can access the survey to answer it, in particular as people without an easy, cheap, and reliable access to Internet may have a specific profile (e.g. people from developing countries, people from disadvantaged backgrounds, people in remote places or at sea for long periods of time, possibly people in prison, etc.).

If the survey will be used to influence decisions that will affect those people, there is obviously an ethical issue to consider if this coverage error is not addressed.