How to sample a statistic?

Question

Disclaimer: I am a software developer and I like stats but I'm not a professional statistician, I already experienced that my wording is not always the correct jargon. Please keep in mind.

I would like your opinion on how to build a concrete statistical model:

The Goal Why am I doing this: I need to assess the reliability of a food database. Is the food present that most of the people eat? With the help of CrossValidated I made an arbitrary benchmark that represents what I want to know:

1. The top 30% food items
2. by amount a normal person eats in one meal
3. sold in Austria annually

The Target Group The goal is to understand what are the most eaten foods in Austria (citizens: 8m), but more by People that are Internet savvy, therefore I tend to say urban, educated. There is a special segment Mother with Children and Athletes that I will address in a second study.

The Data It is about branded food because the database is exhaustive with natural food. Each Brand has to be recorded separately.

I would like to know what you think of my approach:

I go to a supermarket and observe the cash desk for an amount of time and record every food that has been bought. This is one sample the population is the overall consumption of items per food. e.g. 23000 liters of coca-cola, 600000 apples

1. Is this a wise design?
2. What would be the appropriate sample size?
3. How would you do it? Take into account this is currently a hobby we are doing in 3 persons. I'm comfortable to spent a few hundred Euros on the inquiry but only if spent wisely.

Answer 1

Sampling Method

I would like to expand the problem mentioned by Henry. Let's just point out a few problems that may arise:

• People tend to use the supermarket that is close to their homes. As you know, there are different types of people in different areas - and I would expect that the goods bought highly depend on personal education and financial background.
• There are supermarkets with higher prices - it is likely that those supermarkets are visited by different people than the discount supermarkets.
• Mothers with children probably buy at different times than people that do work full-time. But do they buy the same?

In statistical terminology, you will have to take care of the sampling method you use. With every random sample you can make a guess at the distribution it was taken from. However, you have to take care that you take the sample from the correct distribution and not a special subset.

Sample sizes

Firstly, on your wording: The population we are talking about is the set of all items that can be bought, not the people living in austria. The population always denotes the possible outcomes of one random sample - and you are observing items bought.

It is hard to tell if you will get enough samples - this will depend on the number of customers you will be able to observe, as well as it will depend on the amount of different goods you record.

Let's have a look at two (very constructed) examples. Say, you record 1000 people each buying exactly one item. In extreme cases, the following might happen:

• All people buy the same product, let's say it is milk. In this case, your sample size should certainly be big enough to conclude that milk is one of the top sold products.
• Everybody buys something different. Then, with this sample size, it will be impossible to determine the most sold product.

This shows that the sample size you have to take on a huge amount depends on the variance you encounter in your data.

Furthermore, the sample sizes depend on the statistical method you will use. Usually, the more you can assume on your distribution, the stronger the method you can use. For example, if you can assume a normal distribution, you may use parametric tests that usually do not need a lot of samples. This is not surprising, as you put a lot of information (normality) as a guess in your data, which leaves only a little bit of information to be determined by the data. However, if you have no information on the distribution, the test will have to guess everything. This naturally means more information will be needed beforehand.

That is why often small sample sizes are taken as a pre-study. Afterwards, you will have a feeling on the variance and will be able to determine the statistical methods that will be used as well as their requirements in terms of sample sizes.

Further Additions

Finally, you should define the groups you are looking for. Is the manufacturer of something important to you? Will you just group 'Cheese', or will there be different groups of cheese?

How would I do it? This really depends on my intention. Do I have a budget? Do I have multiple people taking samples? Maybe there are supermarkets that offer me their product statistics. Maybe it would be an idea to interview the people you recorded to identify differences in personal background. Then you could check whether this differences influence the output. Furthermore, it might be worth doing a small study first to identify further problems that may arise with sampling and data recording.

As you are looking at Austria, I assume you speak German. Which means I can point you to a book that I do not yet have read in total but that might bring up a lot of questions relevant to your problem. It is called "Stichproben" by Kauermann and Küchenhoff. Sorry for all the english readers around here, I do not know an english book about that topic...

Answer 2

Sorry for the late answer (and after an answer has been accepted ;-)), but there is an issue that is flawed in the proposal and perhaps in the answers. If you are asking about foods that are eaten over an annual range, then any sample that does not encompass the year may well miss (or, conversely, over-weight) some seasonal anomalies.

For instance, a sample that is constructed during holidays or festivals may overcount alcohol, sweets, breads, certain types of meats, etc. Depending on the season, different vegetables may be under or over counted.

Were I undertaking such a study, I wouldn't do it observationally, but instead inquire about the sales of the different supermarket chains. This would give you a far larger dataset than could be obtained visually.

A more out of the box approach would be to consult with certain tax agencies or the food inspection agencies; these may have very good knowledge of the quantity and dollar volume of sales. Another out of the box approach is to have a raffle for people who mail their store receipts. Of course this is a biased sample (e.g. some people may not want to share indications of their addiction to snack foods), but it is one method of getting post-sales info.

Last, but not least, think about what will be the goal of the user of this database. If they feel that there is a mismatch between your sampling methods and their needs, then either the design will need to be adjusted or they will need to be educated on the advantages of your design.

Answer 3

This may be rather a comment to the first point under Sampling Method of Thilo Schneider's answer (but it's too long and I cannot format nicely):

Not only do different groups of people buy different things, but the supermarkets do adjust their goods to that, so there is a reinforcement of the bias. People cannot buy their preferred brand in the supermarket close by - I for one in that situation either switch to something else or postpone buying it till I happen to come along some other shop where I can buy it.

Conclusion: If you want to compare preferences (e.g. which brand/kind of cheese is eaten most) and taking into account that you have limited resources: pick supermarkets with high "local diversity", i.e. either supermarkets that offer a lot of choice themselves, or supermarkets from area where lots of supermarkets are close by.

In other words, if I wanted to compare, I'd try to have the consumers as free choice as possible, and rather account for the bias in picking supermarkt locations by stating afterwards that these results are for that kind of supermarket and location. Though it depends on the aim of your study whether this is a valid approach.