Questionnaire design

Answers to a questionnaire - Frequency of activity X

• 1 - 5
• 6 - 10
• 10 - 20
• 20 or more

I have two questions regarding these answers:

• Uneven bins, the answer 10 to 20 has twice the number of numerical answers, if you were to analyse this data after collection, what implications would this have?
• Similarly the final answer, an open ended upper bound. I am advised to estimate the average for this category, I disagree as we wont know the upper limit of the answer.

Your thoughts on this would be greatly appreciated.

• I don't think you have a way to estimate the average for the upper category unless you make an assumption about the distributional form. – Glen_b Sep 18 '13 at 10:23
• Thank you. An assumption about the distribution of answers for all? – Will Sep 18 '13 at 10:31
• As opposed to the distribution of answers which fall into that category? – Will Sep 18 '13 at 10:38
• Either would work -- if you do it only for the last category you'd have to assume the parameter values; if you do it for all categories you can use the values you have to estimate the parameters (you could use techniques that deal with censoring). – Glen_b Sep 18 '13 at 12:00
• This is what I thought. Do you have any insight on uneven intervals when the distribution is unknown? – Will Sep 18 '13 at 12:18

a) Uneven bins, the answer 10 to 20 has twice the number of numerical answers, if you were to analyse this data after collection, what implications would this have?

The first implication is I'd probably laugh out loud at the person who designed the questionnaire (do it internally if the person is close by), and secretly put him/her under my crazy list.

The second implication is to remember last time I tried to use red wine vinegar when the recipes was calling for red wine and that didn't turn out so well. So, I'll just man up, take this as an ordinal variable, and try not to be overly creative with substituting anything.

b) Similarly the final answer, an open ended upper bound. I am advised to estimate the average for this category, I disagree as we wont know the upper limit of the answer.

That depends on who advised me to do that. If it's an unknown reviewer, I'd just scan the pages about central tendency from any intro statistics book, circle "median" or "mode" and send that back the median or mode.

If it's someone in power like my boss (who is already on my crazy list,) I'll probably suggest some distributions (I'd start with Poisson, but it's up to anyone's guess,) simulate with an array of parameters (e.g. lambda) and categorize the results according to the weird categorization until the percentages agree. The catch is that the result is going to be wrong cause it's hard to find a behavior that is only influenced by one distribution.

Also, where is the answer for 0 activity?

Why would you laugh out loud? Yes internally!

Please kindly forgive my humor. In many case this kind bastard children between a continuous and an ordinal variables just make me feel sad and funny the same time. We really didn't save too much respondents' burden in answering the question, but we kind of put in a difficult situation later. Should the 10-20 were broken down by an increment of 5, and if 20+ has a very small count, some kind of "trimmed mean" can still be a convincing alternative.

And speaking about trimmed mean, if not many of them are in 20+ (say, <2%), you may actually consider trimmed mean. You'll still need to approximate the mean in each bracket, but at least it's more reasonable.

Would you suggest using an upper bound for the final answer?

Depends on what the activity is. If not too much burden I'll ask for the actual number. If it's too much burden, I'd revise the category so that perhaps no more than a few percent of people will choose the top category.

• Great insight, many thanks. Why would you laugh out loud? Yes internally! – Will Sep 18 '13 at 13:59
• Would you suggest using an upper bound for the final answer? – Will Sep 18 '13 at 14:04
• Oh - answer for 0 activity, there is one in the original question, I appear to have omitted it whilst re-writing it – Will Sep 18 '13 at 14:13