4
$\begingroup$

In chapter 2 ESL book authors write: Let's look at example of linear model in a classification contextenter image description here

They fit a simple linear model $g = 0.3290614 -0.0226360\cdot x_1 + 0.2495983 \cdot x_2 + e$,

where $g$ is given with values $0$ or $1$. Then they made a decision boundary where $\hat{G}$ is Orange, if $\hat{y}>0.5$ , otherwise it is blue. Question: There is a separation line on the picture. Where did intercept and slope for this line come from?

In the ElemStatLearn R package, they simply put as
abline( (0.5-coef(x.mod)[1])/coef(x.mod)[3], -coef(x.mod)[2]/coef(x.mod)[3])
where the first term is the intercept, and the second term is the slope of this line.

$\endgroup$
1
  • $\begingroup$ Could you maybe answer in intuitive way the following question.Figure 2.1 can be viewed as representation of 3d plot where the color of points simply represents a third dimension. There is a cloud of point where majority z=0; and there is another cloud where majority of z=1. By building linear model one simply builds the plain separating two clouds of points. Question, how does this plain projects on 2d plot? Or is it not, and decision boundary line in 2d plot has nothing to do with plane in 3d plot? $\endgroup$
    – den2042
    Sep 5, 2014 at 22:21

1 Answer 1

2
$\begingroup$

$\hat{y}$ is evaluated with the linear model $g = 0.3290614 − 0.0226360 \cdot x_1 + 0.2495983 \cdot x_2$; The decision boundary is at $g = 0.5$.

This is a linear equation with two parameters ($x_1$ and $x_2$), so it is an equation for a straight line. To get the intercept and slope, we can solve for $x_2$:

$$ 0.5 = g(x_1, x_2) \\ 0.5 = 0.3290614 − 0.0226360 \cdot x_1 + 0.2495983 \cdot x_2 \\ 0.2495983 \cdot x_2 = 0.5 - 0.3290614 + 0.0226360 \cdot x_1 \\ x_2 = (0.5 - 0.3290614)/0.2495983 + 0.0226360/0.2495983 \cdot x_1 $$ So, interpreted as a function $x_2(x_1)$, you get the intercept $(0.5 - 0.3290614) / 0.2495983 \approx 0.685$ and the slope $0.0226360 / 0.2495983 \approx 0.091$ for the boundary. This agrees with the image (assuming the axis run from 0 to 1, as indicated by the 0.5 in the R-code for centering).

Also note the similarity to the R code, e.g. coef(x.mod)[3] corresponds to the coefficient of $x_2$ in $g$, $0.2495983$.

$\endgroup$
2
  • 1
    $\begingroup$ Let me add a question: by what criteria was it chosen $g=0.5$ ? $\endgroup$
    – Fam
    Sep 17, 2019 at 23:55
  • $\begingroup$ It's the most obvious boundary value between the values 0 and 1, as it's right in the middle. $\endgroup$
    – dobiwan
    Apr 17, 2020 at 13:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.