# How to find "dependency" between categorical variables and the dependent variable, also categorical

Please edit the title if the wording is incorrect

Imagine to have a dataset of recipes that works like this: each row is a recipe, where the ingredients are predictors and the dependent variable would be the cuisine (i.e.: Italian, Chinese, Mexican, ...). E.g.:

   recipe |  ingredient 1  |  ingredient 2  |    ...    |   cuisine
----------+----------------+----------------+-----------+-------------
pizza  |     dough      |     tomato     |    ...    |   italian
----------+----------------+----------------+-----------+-------------
burrito |  bread thingy  |    guacamole   |    ...    |   mexican
----------+----------------+----------------+-----------+-------------
carbonara |   spaghetti    |      eggs      |    ...    |   italian
----------+----------------+----------------+-----------+-------------


and so on...

Logically I can imagine that recipes that include "pasta" and "ragù" are more likely to be Italian, whereas recipes that have "rice" and "soy sauce" could be more easily Chinese. Conversely, if I have "eggs" then I expect that there's no particular cuisine associated to it because all the world uses eggs.

I'm looking for a way to show these relations, to answer questions like: "what are the 2 most important ingredients in Italian cuisine?" or "if I have avocado and garlic, which is the most likely cuisine associated to them?".

I know that the first step would be to one-hot-encode the ingredients, but then I'm lost. I've looked for answers here and elsewhere but most I could find was related to continuous variables.

Keep in mind: I'm not trying to make a ML model for this, I'm just trying to figure out the relationships between ingredients and cuisine. Also, my dataset is much simpler than this (I only have 5 predictor variables with limited choices), but the logic is exactly the same as described.

Btw, not relevant to the question per se, but I'm using python, so if you already have a pre-cooked solution to this predicament feel free to share.

You can use Bayesian inference to solve your problem.

If you answer the question "what is the proportion of Italian recipes?", you have a prior probability of Italian recipe $$P(Italian)$$. Also, you can get prior probability of having tomato in recipe $$P(tomato)$$.

Then, you can calculate $$P(Italian|tomato)$$ as the proportion of Italian recipes among the recipes that contain tomatoes. In the same way you can get $$P(tomato|Italian)$$. There are posterior probabilities. By default they should be calculated using Bayes formula: $$P(Mexican|tomato) = \frac{P(tomato|Mexican)P(Mexican)}{P(tomato)}$$

You can get it directly from your data.

If you want to know what is the most probable cuisine associated with tomato, you can compare $$P(Italian|tomato)$$, $$P(Mexican|tomato)$$, $$P(other\_cuisine|tomato)$$ and choice the max one:

$$answer = \underset{cuisine}{\operatorname{argmax}} P(cuisine|tomato)$$

Also, you can combine any number of hypothesis in Bayes formula, but it can be a little tricky at first:

$$P(cuisine|ingr_1, ..., ingr_n) = \frac{P(ingr_n|cuisine)P(cuisine|ingr_1, ..., ingr_{n-1})}{\sum_{cuisine}^{C}P(ingr_n|cuisine)P(cuisine|ingr_1, ..., ingr_{n-1})}$$

where

$$C = \{Italian, Mexican, other\_cuisine, ...\}$$

• Hi, thanks for the answer. I am somewhat familiar with Bayes inference, but I'm not exactly sure about the last formula, especially the part above where you have P(cuisine | ingr1, ... ingrn). How does it connect with the rest? Jun 2, 2020 at 0:16
• @wtfzambo If you want to know probability of a particular cuisine given more than one ingredient (your example: if I have avocado and garlic, which is the most likely cuisine associated to them?), you need to use Bayes formula for multiple observations - the last formula. The "simple" one can handle only one observation (ingredient). In general you need to calculate the probability recursively, by taking into account additional observation on the each step. Jun 2, 2020 at 15:04
• Ok I understood, I didn't know one could apply Bayes theorem recursively. Thanks, I guess I'll start with the simple inference and then eventually move on to the tricky part if I need. Jun 3, 2020 at 9:05