How to sample a statistic?

Disclaimer: I am a software developer and I like stats but I'm not a professional statistician, I already experienced that my wording is not always the correct jargon. Please keep in mind.

I would like your opinion on how to build a concrete statistical model:

The Goal Why am I doing this: I need to assess the reliability of a food database. Is the food present that most of the people eat? With the help of CrossValidated I made an arbitrary benchmark that represents what I want to know:

1. The top 30% food items
2. by amount a normal person eats in one meal
3. sold in Austria annually

The Target Group The goal is to understand what are the most eaten foods in Austria (citizens: 8m), but more by People that are Internet savvy, therefore I tend to say urban, educated. There is a special segment Mother with Children and Athletes that I will address in a second study.

The Data It is about branded food because the database is exhaustive with natural food. Each Brand has to be recorded separately.

I would like to know what you think of my approach:

I go to a supermarket and observe the cash desk for an amount of time and record every food that has been bought. This is one sample the population is the overall consumption of items per food. e.g. 23000 liters of coca-cola, 600000 apples

1. Is this a wise design?
2. What would be the appropriate sample size?
3. How would you do it? Take into account this is currently a hobby we are doing in 3 persons. I'm comfortable to spent a few hundred Euros on the inquiry but only if spent wisely.
-
certainly an interesting approach. Obviously, the longer you can observe the less uncertainty you'll have (due to limited sample size) and the more such observations you can make (e.g. different days of week, hours of the day, months of the year, different locations and stores), the less biased the observation will be. –  Andre Holzner Sep 17 '11 at 15:27
what is the 30 percentile of the most eaten foods ? Note that in your example you seem to be comparing apples with litres. Summing the weights would make them comparable. So do you mean 'the top 30% items by total weight sold' ? –  Andre Holzner Sep 17 '11 at 15:34
There will be problems here about how representative your sample is for food in Austria: people may buy different foods elsewhere, for example in specialist shops or markets, and may buy different foods on different days or at different times. –  Henry Sep 17 '11 at 15:49
@Andre Holzner, I added your suggestion 'the top 30% items by total weight sold' to the question –  Roland Kofler Sep 17 '11 at 17:03
I think supermarkets are really representativ and people buy there food mainly there, the occasional biomarket visit etc. does not bother me. And I am more concerned with processed/ branded food than natural food like an apple –  Roland Kofler Sep 17 '11 at 17:05

Sampling Method

I would like to expand the problem mentioned by Henry. Let's just point out a few problems that may arise:

• People tend to use the supermarket that is close to their homes. As you know, there are different types of people in different areas - and I would expect that the goods bought highly depend on personal education and financial background.
• There are supermarkets with higher prices - it is likely that those supermarkets are visited by different people than the discount supermarkets.
• Mothers with children probably buy at different times than people that do work full-time. But do they buy the same?

In statistical terminology, you will have to take care of the sampling method you use. With every random sample you can make a guess at the distribution it was taken from. However, you have to take care that you take the sample from the correct distribution and not a special subset.

Sample sizes

Firstly, on your wording: The population we are talking about is the set of all items that can be bought, not the people living in austria. The population always denotes the possible outcomes of one random sample - and you are observing items bought.

It is hard to tell if you will get enough samples - this will depend on the number of customers you will be able to observe, as well as it will depend on the amount of different goods you record.

Let's have a look at two (very constructed) examples. Say, you record 1000 people each buying exactly one item. In extreme cases, the following might happen:

• All people buy the same product, let's say it is milk. In this case, your sample size should certainly be big enough to conclude that milk is one of the top sold products.
• Everybody buys something different. Then, with this sample size, it will be impossible to determine the most sold product.

This shows that the sample size you have to take on a huge amount depends on the variance you encounter in your data.

Furthermore, the sample sizes depend on the statistical method you will use. Usually, the more you can assume on your distribution, the stronger the method you can use. For example, if you can assume a normal distribution, you may use parametric tests that usually do not need a lot of samples. This is not surprising, as you put a lot of information (normality) as a guess in your data, which leaves only a little bit of information to be determined by the data. However, if you have no information on the distribution, the test will have to guess everything. This naturally means more information will be needed beforehand.

That is why often small sample sizes are taken as a pre-study. Afterwards, you will have a feeling on the variance and will be able to determine the statistical methods that will be used as well as their requirements in terms of sample sizes.

Finally, you should define the groups you are looking for. Is the manufacturer of something important to you? Will you just group 'Cheese', or will there be different groups of cheese?

How would I do it? This really depends on my intention. Do I have a budget? Do I have multiple people taking samples? Maybe there are supermarkets that offer me their product statistics. Maybe it would be an idea to interview the people you recorded to identify differences in personal background. Then you could check whether this differences influence the output. Furthermore, it might be worth doing a small study first to identify further problems that may arise with sampling and data recording.

As you are looking at Austria, I assume you speak German. Which means I can point you to a book that I do not yet have read in total but that might bring up a lot of questions relevant to your problem. It is called "Stichproben" by Kauermann and Küchenhoff. Sorry for all the english readers around here, I do not know an english book about that topic...

-
A really great answer. I will edit my original post to answer the flaws you pointed out and define my topic better. –  Roland Kofler Sep 17 '11 at 18:47
you said ": It is hard to tell if you will get enough samples" given the population of 8m what would be the approrpiate sample size? –  Roland Kofler Sep 17 '11 at 19:15
I made a rather large edit to include some of the information you gave. Hopefully, this gives you some more ideas and hints. –  Thilo Sep 18 '11 at 7:38
Thilo, thank you for the tip, I've already started to read the Book on Google Books, I did a few things in R so I really enjoy it. About budget and people: I've already addressed this in the last sentence of my Question. Also, I will look at the specific brand, because I need to know I that product "m&m's", "philadelphia by kraft" etc. is in my DB. I imagine this will increase the sample size needed. –  Roland Kofler Sep 18 '11 at 9:32
I will reedit my Question concerning distribution during the day. I currently believe it will be a zipf-law distribution not a normal. But I still want to do some research. What do you think am I on track with zipf? –  Roland Kofler Sep 18 '11 at 9:32

Sorry for the late answer (and after an answer has been accepted ;-)), but there is an issue that is flawed in the proposal and perhaps in the answers. If you are asking about foods that are eaten over an annual range, then any sample that does not encompass the year may well miss (or, conversely, over-weight) some seasonal anomalies.

For instance, a sample that is constructed during holidays or festivals may overcount alcohol, sweets, breads, certain types of meats, etc. Depending on the season, different vegetables may be under or over counted.

Were I undertaking such a study, I wouldn't do it observationally, but instead inquire about the sales of the different supermarket chains. This would give you a far larger dataset than could be obtained visually.

A more out of the box approach would be to consult with certain tax agencies or the food inspection agencies; these may have very good knowledge of the quantity and dollar volume of sales. Another out of the box approach is to have a raffle for people who mail their store receipts. Of course this is a biased sample (e.g. some people may not want to share indications of their addiction to snack foods), but it is one method of getting post-sales info.

Last, but not least, think about what will be the goal of the user of this database. If they feel that there is a mismatch between your sampling methods and their needs, then either the design will need to be adjusted or they will need to be educated on the advantages of your design.

-