# Can statistics based on crude units of measure be improved?

I'm collecting online recipes in order to calculate average ingredient proportions for, say crepes or pastry. Most volume based units of measure in english recipes (cups, tablespoons, teaspoons) differ from country to country and sometimes differ within the same country depending on whether traditional, metric or government standardised units are used.

I have used my own guesses to convert from these units to milliliters. So, for example, if it is unclear exactly which cup measure is indicated, I use a metric cup (250ml).

I wonder if there are tools that can be used to infer the most likely conversion for these crude units given that a percentage of the data is recorded using more accurate units (grams, milliliters). Or, failing that, to account for the uncertainty in some way (larger confidence intervals?).

Update: I see that some questions have comments asking what the stats are used for. In case it is helpful, I'm using them to answer questions like "How do crepes differ from Swedish pancakes?" and "How do English recipes for crepes differ from French recipes?". For example...

• The "average ingredient proportions" may be nonsensical in some cases; an obvious example being that to make 8 times the amount of food a recipe describes you don't just octuple all the ingredients mentioned (eg. salt). In addition, I would be careful to make sure that the averaging concerns "similar recipes"; if one recipe contains an ingredient that the other does not that extra ingredient might chance the composition of the mixture substantially. Apologies for the non-mathematical comment but I do feel you are facing a scenario that can potentially grossly abuse Stats. – usεr11852 says Reinstate Monic Nov 4 '12 at 3:14
• Why are you guessing? There are exact conversions between English standard and metric measurements. Sounds like a fun project, though... – gung - Reinstate Monica Nov 4 '12 at 3:19
• @user11852 There is a sense in which I'm just playing around here, so I'm not ready to make any grand claims for the stats so far produced. To answer one concern, I have filtered out recipes where substituted or additional ingredients may conceivably make a difference. – Chris Steinbach Nov 4 '12 at 4:36
• @gung I'm not guessing the conversions. It is the measure itself that is a guess. For example, I often can't tell whether a cup refers to a US cup or an imperial cup (284ml). If I determine that a US cup is meant, then it might not be clear whether a traditional cup (236.58ml) or a 'legal' cup (240ml) is meant. – Chris Steinbach Nov 4 '12 at 4:49
• Couldn't you convert each measure into the proportion of the recipe's total volume? I think you would need to do that to account for batch size anyway (i.e., one recipe might be for 6 pancacks and another for 12). I don't know how you would deal with eggs however. Is a French recipe refering to a large egg like an American recipe might? Is a large egg the same volume across countries? – RioRaider Nov 4 '12 at 5:57

You could certainly arbitrarily assign outliers to have the conversion system that reduces their outlier-ness. But I think this would be more of an arbitrary decision than an improvement.

You may also be seeking a simple test for whether your distribution of cups of a given ingredient in a given recipe category is unimodal or not (after normalizing for servings and other ingredients). Hartigan’s dip test is one such test, but I doubt it will be helpful here. You could certainly make a simulated best-case scenario dataset to run the diptest on and get a better sense of its utility here.

My inclination is to treat this as a data mining problem in which we seek to find which other features in the recipe are associated with the different kinds of cups. I would hypothesize that cup type is associated with region/culture and that region/culture is best identified by linguistic features in the recipes.

I would to train a classifier, perhaps just Naïve Bayes, with a dataset in which the cup type is known. This may rule out online recipe websites, many of which automatically translate from metric to imperial units. Ideally, you would scrape the contents of several regional cookbooks.

I’d probably run a quick sanity check to see if there actually was any regional variation in units. Who knows, variation in measurement units may be better explained by whether the recipe is targeted to casual cooks (unlikely to own scales) or serious ones.

Ratios of ingredients may be helpful, but it is perhaps too complicated to be worth pursuing. Most recipes are tolerant of imprecision. Within less tolerant categories (say cakes) the crucial factor is not the raw ingredients, but the ingredients of the ingredients. That is, fat, protein, etc. So you would ideally lookup ingredients’ nutritional profiles in any of the many online sources.

I think it would probably be easier and more successful to see if we can classify based on the words in the recipes, including the title, ingredients, or even step-by-step instructions. For example, “caster sugar” (British English) vs. “fine sugar” (American) or “self-raising flour” (a British product).

To identify these shibboleth features, you might train random forests to predict region or measurement unit system when given a TF-IDF (term frequency – inverse document frequency) matrix made by counting words in recipes. In R, the varSelRF and Boruta packages are useful for variable selection via random forest.

But I do think the key is obtaining a data set with known ground truth.

• I'm not familiar with Hartigan's dip test. I'll read up on it in case it could help. Unfortunately my current data set has been separated from all recipe text, but I think a classifier is a great idea for the future. Thanks for the detailed answer. – Chris Steinbach Nov 12 '12 at 10:24