# Categorical variable affecting the entire model

I have a linear model with one continuous dependent variable ($y$) and one explanatory variable ($x$). The model: $y = b_0 + b_1 * x$; now I and need to add a categorical feature to it. The problem is that training for each category should result in its own curve ($b_0, b_1$ pair). Each category is a different process. So I cannot just use a linear regression and add just add $b_2*x_2$.

I'm tempted to use the simplest way: train a separate model for each process. But my feeling is this might fall in one of well-known problems as a single model. The actual problem is in ML field.

• Let the categories be $x_c \in \{1, 2, 3\}$ then you could define $x_1 = 1$ if $x_C = 1$ and $x_1=0$ otherwise, $x_2= 1$ if $x_c = 2$ and $x_2 = 0$ otherwise. then use $y=\beta_0 + \beta_1 x+\beta_2 x_1 + \beta_3 x_2$. Would that be an option? – dietervdf Jan 15 '17 at 2:47
• For each $x_c I need continuous observable$x and a separate both $b_0 and$b_1. In your example only one $b_x coefficient will be added to the sum for each$x_c. Utilizing your idea I can say: for $z_c from (1, 2, 3), if$z_c = 1 – VladimirLenin Jan 15 '17 at 3:09
• in my proposal the intercept will be different for each $x_c$, since when the category is for instance $2$, then the intercept will be $\beta_0 + \beta_3$, while if the category is $x_C=3$ then the intercept equals $\beta_0$. If you want different $\beta_1$ then you would need to add interactions into the model, since the category would change the value of the $\beta_1$ coefficient. – dietervdf Jan 15 '17 at 3:14
• Are you building the model in R? – dietervdf Jan 15 '17 at 3:14
• I'll construct an expanded answer, just a sec :) – dietervdf Jan 15 '17 at 3:22

Let's say your continous variables are the outcome $Y$ and the predictors $X_1$. Now consider a categorical variable $X_C\in \{A,B,C\}$ then you could define the following variables:

$$X_A = \begin{cases} 1 & \text{if } X_C = A\\ 0 & \text{otherwise}\end{cases},\qquad X_B = \begin{cases} 1 & \text{if } X_C = B\\ 0 & \text{otherwise}\end{cases}$$

Next you could consider the model:

$$Y_i = \beta_0 + \beta_1 X_1 + \beta_A X_A + \beta_B X_B + \beta_{1A}X_1\cdot X_A + \beta_{1B}X_1 \cdot X_B + \epsilon_i$$ which would fit your needs, since

• Let $X_C = C$ (the reference category)

Then the model would be: $$Y_i = \beta_0 + \beta_1 X_1 + \epsilon_i$$

• Let $X_C = A$

Then the model would be: $$Y_i = \beta_0 + \beta_1 X_1 + \beta A + \beta_{1A}X_1 +\epsilon_i= (\beta_0+\beta_A) + (\beta_1+\beta_{1A}) X_1+\epsilon_i$$

• Suppose $X_C = B$ then the model would be

$$Y_i = (\beta_0+\beta_B) + (\beta_1 + \beta_{1B}) X_1+\epsilon_i$$

I think this would fit your needs?

• I think this what I need. Could you help with a common name for this device in R/statistics, if any? Btw, it looks it should be $... = (B_0 + B_A) + (B_1 + B_1a) * X_1 + e_i$ for $X_C = A$. – VladimirLenin Jan 15 '17 at 3:53
• woops, typo :), what do you mean by common name for this device? You can implement this rather easy in R by applying as.factor() on the categorical variabel – dietervdf Jan 15 '17 at 4:01
• I mean the name for this should be "non-linear regression"? – VladimirLenin Jan 15 '17 at 4:12
• It is still categorized as linear regression since these are interaction terms. – dietervdf Jan 15 '17 at 4:17