Help me plan/design a survey We are doing a survey regarding what members of a club think about the club. Say we have roughly 1000 members, who will be divided into five groups of 200 in each. Then each group will be given a survey at different times of the year (one group March, one April, and so on).
Besides the fact that people may think differently about the club over the year, and that we make sure that the groups are fairly similar.
Is there anything else statistically (or otherwise) that we need to think of?
 A: 1. Define some domains from which you’d derive questions
For instance, what aspects of the service? What kind of responses or actions would you expect the members to produce?
2. Invite a few small groups of your members (about 4-6) for an informal interview
The technical term is “focus group” if you would like to read up on that. Basically, form some groups consisting of members with bigger difference between groups but similar within groups (for instance, limited to one gender, one particular age group, or one particular membership status). Prior to the meeting, consult the domains you have established and come up with some questions that would spark open-ended discussions. Ideally, you should have one moderator and one recorder, but a digital recorder can also do. If you have big budget, there are companies specialized in this technique; if you have small budget, try contact some faculty members who teach qualitative research skill at your local universities and hire a couple good students to help you; no budget? There are plenty of books and websites about this. You may revise the questions along the way, conduct enough meetings until you feel a clear pattern has emerged and no more new information you can learn, or your time/budget limit is hit.
The purpose of these focus groups is to ensure your later questionnaire will make sense to them, will use their lingo, and will target at the areas that members are concerned.
3. Prepare your questions, and pretest
Draft your questions. One trick I have learned is that don’t write up the whole questionnaire as if you’re doing a composition. Write each question separately. Go back to select the potential candidates and arrange them in a logical order. The format of delivering the questionnaire dictates the kind of questions you can ask. If it’s online, then you probably don’t want to rely exclusively on answers that need to be typed up. If it’s a person-to-person interview, then you may want to enrich your information by adding some open-ended questions. Whichever format it will be, keep your wording neutral. For instance, question like “In a scale 1 to 10 with 1 being the worst and 10 being the best, how would you rate the service of our club?” instead of “In a scale 1 to 10 with 1 being the worst and 10 being the best, how satisfied are you with the service of our club?”
You may invite another few members or at least outsiders that have similar literacy level to look at your draft questionnaire and try filling it in. Ask them to flag places that the questions and answer schemes might have confused them. Address those suggestions by revising the questionnaire.
In formal survey design, there is also a stage to refine the question items (using technique like factor analysis). If you’re operating at that caliber, you may need to consult some specialists. But for a small scale private survey, I believe having logical questions that are valid on the surface should suffice.
4. Further note on interview format
If it’s computer-based or online, keep the layout such as font size, background color, etc. uniform across time points. If it’s a person-to-person interview, try to use the same batch of interviewers in each time points. Train them how to deliver the question and briefly retrain them prior to each follow up interview. Sometimes members may ask questions, so it’s better to provide your interviewers a couple sheets of FAQs so that they can give uniform responses. If you use online questionnaire or computer based questionnaire, it’s also important to trial run your data input and export devices to ensure every component works.
5. Randomly assign members
Don’t “make sure that the groups are fairly similar.” From the get go, randomly assign your 1000 people to each time point and stick to it. If you very much want to make sure there is an equal share of males and females in each time point, randomly assign all the males first, and then females. Same can be done to other variables.
6. Pay attention to attrition rate
Well, the question really doesn't say why longitudinal and what intervention is going to be rolled out. So this part may not apply.
Some club membership can be highly volatile. If your club is one of them, I would suggest reframing your sampling scheme or shorten the duration between time points. The problem is that by month five, people who really hate the club would have gone; the ones who stay are either apathetic or positive towards the club. This can lead to a false impression that the service has improved. I don’t have any statistically supported new scheme for you (others may chime in), but perhaps you should also allow new members who join along this 5 months to be sampled as well. You may still exclude people who were previously interviewed, but at least you can lower the bias I mentioned above.
I originally thought that your club is going roll out some new service and that’s why you’d like to track the response along time… but you also mentioned that “people may think differently about the club over the year.” And this led me to wonder if you’re just treating it as a census? If it is so, I would strongly suggest coming up with a valid sample size and use that result as the representation. Census of all customers seems to be a waste of resources. 
A: Far too many surveys are made without enough attention to the questionnaire and how it may be interpreted. Everything that can be missinterpretted, will be misinterpreted.
Using trial groups to ensure that the questionnaire is read as intended, is one way to get comparable results. 
A: You must take care that the opinion of people are not baised in any sense. like you must not tell members that they have to fill asurvey, because in this case their opinion may get baised because of there discussion thinking with each other or their any pre thinking. 
