Maths Behind Dummy Variable in Linear Regression (One Hot Encoding)?

Question

I understand the logic behind using k - 1 dummy variables for K Categories (multi collinearilty etc) but trying to understand how the math behind it works.

Consider the following Example: We Code:

Female as 1 Male = 0

Regression Eqn: y = B₀ + B₁*Age + B₂*Gender

For Female: y = B₀ + B₁Age + B₂
For Male: y = B₀ + B₁*Age

Here it seems that we don't account for the effect of Gender when he is Male since the term becomes Zero.

Also If I Code categories other way round should I get the same regression Equation.

Answer 1

In your notation $B_2$ describes the difference between the effects of being female and being male. Everything else is in the intercept $B_0$. Consider this example:

Assume a imaginary linear relationship between Age, Gender and Weight. For men, it is Weight = 20 + 2 * Age, while it is for women Weight = 10 + 2 * Age, nevermind the units. Having Female as 1 in a one-hot encoding results in a linear model like this: Weight = 20 + 2 * Age - 10 * Gender. $B_2 = -10$ tells you that for a female (because encoded as 1), the weight is 10 lower. If you reverse the encoding, $B_2$ would have the value $10$, as you now describe the weight increase effect of being male.

Answer 2

We can think of it as we have a model for the male $B_0+ B_1 Age$ and to modify it for female, we realized that we just have to modify by a constant addition of $B_2$.

If you code it in the opposite way, say $G= 1- Gender$,

then we have $$y=B_0 + B_1 Age + B_2(1-G)=(B_0 + B_2) + B_1 Age - B_2 G$$