# Linear regression of 0/1 response (Fig. 2.1 of The elements of statistical learning)

In chapter 2 ESL book authors write: Let's look at example of linear model in a classification context

They fit a simple linear model $g = 0.3290614 -0.0226360\cdot x_1 + 0.2495983 \cdot x_2 + e$,

where $g$ is given with values $0$ or $1$. Then they made a decision boundary where $\hat{G}$ is Orange, if $\hat{y}>0.5$ , otherwise it is blue. Question: There is a separation line on the picture. Where did intercept and slope for this line come from?

In the ElemStatLearn R package, they simply put as
abline( (0.5-coef(x.mod)[1])/coef(x.mod)[3], -coef(x.mod)[2]/coef(x.mod)[3])
where the first term is the intercept, and the second term is the slope of this line.

• Could you maybe answer in intuitive way the following question.Figure 2.1 can be viewed as representation of 3d plot where the color of points simply represents a third dimension. There is a cloud of point where majority z=0; and there is another cloud where majority of z=1. By building linear model one simply builds the plain separating two clouds of points. Question, how does this plain projects on 2d plot? Or is it not, and decision boundary line in 2d plot has nothing to do with plane in 3d plot? Sep 5, 2014 at 22:21

$\hat{y}$ is evaluated with the linear model $g = 0.3290614 − 0.0226360 \cdot x_1 + 0.2495983 \cdot x_2$; The decision boundary is at $g = 0.5$.
This is a linear equation with two parameters ($x_1$ and $x_2$), so it is an equation for a straight line. To get the intercept and slope, we can solve for $x_2$:
$$0.5 = g(x_1, x_2) \\ 0.5 = 0.3290614 − 0.0226360 \cdot x_1 + 0.2495983 \cdot x_2 \\ 0.2495983 \cdot x_2 = 0.5 - 0.3290614 + 0.0226360 \cdot x_1 \\ x_2 = (0.5 - 0.3290614)/0.2495983 + 0.0226360/0.2495983 \cdot x_1$$ So, interpreted as a function $x_2(x_1)$, you get the intercept $(0.5 - 0.3290614) / 0.2495983 \approx 0.685$ and the slope $0.0226360 / 0.2495983 \approx 0.091$ for the boundary. This agrees with the image (assuming the axis run from 0 to 1, as indicated by the 0.5 in the R-code for centering).
Also note the similarity to the R code, e.g. coef(x.mod)[3] corresponds to the coefficient of $x_2$ in $g$, $0.2495983$.
• Let me add a question: by what criteria was it chosen $g=0.5$ ?