Linear regression with 2 different categorical variables

Question

I'm confused how the data is stored for 2 categorical variables, consider the example where we are testing the response of y with whether someone says yes or no, with simple formula $y = \alpha + B_{1}x_{1}$
I understand that 'yes' is represented as $x_{1} = 1$, so $y = \alpha + B_{1}$ and 'no' would be represented by $x_{1} = 0$ and $y = \alpha$,
My issue arises when we have more than 1 categorical variable.
Suppose on top of this example we now have another predictor independent of 'yes' and 'no', for example say 3 levels, cold, medium, hot. Our new model should look like
$y = \alpha + B_{1}x_{1} +B_{2}x_{2} +B_{3}x_{3}$
where cold is when x2 and x3 are both 0.
Does this imply the intercept is now shared between 'cold' and 'no'? And how can we obtain 'cold' separately? Would it still be possible to obtain 'cold' separately? Because we can set x2 and x3 = 0, but then if we choose x1 = 1 we are saying that 'yes' is true but I want to hold the {yes, no} variables constant in this case. If we are setting x1, x2, x3 = 0, what this is saying is that 'no' is selected and 'cold' is selected, is there any way for just 'cold' to be selected now? How would one interpret the intercept should it be statistically significant now?

Any help would be appreciated.

Answer 1

Nice question! It's good practice to state your models by including the error term. For example, your first model should be stated as:

$y = \alpha + \beta_{1}x_{1} + \epsilon$.

To make my answer easier, I will assume that $x_1$ stands for gender such that $x_1 = 0$ for males and $x_1 = 1$ for females. I will also assume that $y$ stands for income.

With this in mind, the above model can be viewed as a collection of two sub-models: one sub-model for males and the other for females.

Sub-model for males: $y = \alpha + \epsilon$

Sub-model for females: $y = \alpha + \beta_{1} + \epsilon$

Imagine that you are in a setting where you are interested in determining whether there is a difference in average income between male and female employees at a large company. The sub-model for males states that the average income for male employees of the company is equal to $\alpha$. The sub-model for females states that the average income for female employees is equal to $\alpha + \beta_{1}$. The difference in average income between female and male employees of the company is $\alpha + \beta_{1} - \alpha = \beta_{1}$.

As you can see, including a single dummy variable for gender in your model divides your target population - in this example, employees at a large company - into 2 sub-populations: males and females. For each sub-population, you postulate a model which describes the average value of your outcome variable y (income) as a function of gender.

When you include multiple dummy variables in your model, you end up with more sub-populations and hence more sub-models. For example, if $x_2$ and $x_3$ are dummy variables used to encode the level of education of an employee (highschool, graduate or post-graduate), you would have 2 x 3 = 6 sub-populations and hence 6 sub-models if you were to include the dummy for gender and the dummies for education level in your full model:

$y = \alpha + \beta_{1}x_{1} + \beta_{2}x_{2} + \beta_{3}x_{3} + \epsilon$.

Let's say that $x_{2} = 1$ for graduate education and 0 else; $x_{3} = 1$ for post-graduate education and 0 else. Then your sub-models would be as follows.

Sub-model for males with high-school education: $y = \alpha + \epsilon$

Sub-model for males with graduate education: $y = \alpha + \beta_2 + \epsilon$

Sub-model for males with post-graduate education: $y = \alpha + \beta_3 + \epsilon$

Sub-model for females with high-school education: $y = \alpha + \beta_1 + \epsilon$

Sub-model for females with graduate education: $y = \alpha + \beta_1 + \beta_2 + \epsilon$

Sub-model for females with post-graduate education: $y = \alpha + \beta_1 + \beta_3 + \epsilon$

From the above, you can see that $\alpha$ represents the average income of males with high-school education. So $\alpha$ has a specific interpretation in your full model, which is more easily apparent if you refer to the sub-model for males with a high school education. If you perform a test of the hypotheses:

Ho: $\alpha = 0$ vs Ha: $\alpha \neq 0$

in your full model, you are simply testing:

Ho: average income of male employees with high school education is zero vs Ha: average income of male employees with high school education is different from zero

These hypotheses are not meaningful in this example, as you can't have an average income of zero or else you would be starving. But you get the idea.

What is more meaningful in this example is to test hypotheses like:

Ho: $\beta_1 = 0$ vs Ha: $\beta_1 \neq 0$

Then you would be testing hypotheses like:

Ho: there is no difference in average income between female and male employees with the same education level

vs

Ha: there is a difference in average income between female and male employees with the same education level.