Help understanding Linear Model in ESL book

Question

Also known as "I slowly try to understand ESL (Elements of Statistical Learning)", part two (see part one) Help me understand this (bullets added)

The term $\hat{β}_0$ is the intercept, also known as the bias in machine learning. Often it is convenient to include the constant variable 1 in $X$, include $\hat{β}_0$ in the vector of coefficients $\hat{β}$, and then write the linear model in vector form as an inner product $\hat{Y}= X^\top\hat{β}$ where $X^\top$denotes vector or matrix transpose (X being a column vector). Here we are modeling a single output, so $\hat{Y}$ is a scalar; in general $\hat{Y}$ can be a $K$–vector, in which case $β$ would be a $p \times K$ matrix of coefficients. In the $(p + 1)$-dimensional input–output space, $(X, \hat{Y} )$ represents a hyperplane.

1. If the constant is included in $X$, then the hyperplane includes the origin and is a subspace;
2. if not, it is an affine set cutting the $Y$-axis at the point $(0, \hat{β}_0)$.

From now on we assume that the intercept is included in $\hat{β}$.

My questions:

1. $K$ in the context of a $K$-vector and $p \times K$ matrix of coefficients -- that value is obviously different than $p$; but is $K$ different than $N$ -- the number of observations?
2. What does the notation $(X, \hat{Y} )$ mean?
3. How do they mean "hyperplane"? For example, in the the 2-D example?
4. How do they mean "subspace"? For example, in the the 2-D example?
5. How do they mean "affine set"? For example, in the the 2-D example?
6. When they "Assume the intercept is included" -- which did they choose, option 1 or option 2?
7. How do they mean "$\hat{β}_0$ is the bias" -- why is that word used? Is it related to bias vs variance?
8. Is there indeed a typo as suggested in the quoted part here ; should it be (... in which case $\mathbf{\hat{β}}$ would be a $p \times K$ matrix of coefficients...) In other words, when can $β$ take his hat off ?

Guess at answers:

1. Yes, $p$ is number of columns/variables (i.e. age, weight) $N$ is the number of rows/observations (i.e. Andy, Olly -- though this linear model operates on one-row-at-a-time) , then $K$ is yet another (orthogonal?) axis (i.e. Andy's age, and weight at age 3, age 10, age 20)?
2. It looks like a Cartesian coordinate, but it's generic (uses $X$ and $\hat{Y}$). In 2-D , (1,2) represents the point on a 2-D graph. So does (x,y) represent a set of points; i.e. a line? I have trouble reconciling a scalar $x$ with a vector $X$
3. hyperplane; They mean it cuts the (..."space"?) into two parts. In 2-D , a line cuts (.... ${\rm I\!R^2}$ ?) space on a graph into two separate portions. Indeed, that's the whole point of the "linear model" binary classification (two parts) ; could be called a "hyperplane model" for higher dimensions.
4. subspace; not sure. How can I think of "If the constant is included in $X$", in 2-D space, where X is just a scalar? Do they mean the hyperplane coincides with a plane formed by the intersection of p dimensions? In 2-D, like a line x=0 or y=0?
5. affine set; they mean it does not have an origin? Because the "intercept" has moved it away from the origin, like the "$b$" in $y=mx+b$ ? I am naive about "affine":
6. They've gone with option 1; where the constant ($1$) is included in $X$ and the intercept ($\hat{β}_0$) is included in $\hat{β}$
7. "Bias" and "weight vector" are two relevant terms here explained at the link... I am trying to understand why they would use the word "bias"; In 2-D space the "bias" the y-intercept... is it because when "x" is 0, we know we can't actually estimate "y", but a bias suggests there is some non-zero value for "y"? It is different than bias vs variance (...?)
8. Yes, it should be $\mathbf{\hat{β}}$ -- we only remove the hat when we start "viewing this as a function" (i.e. $f(X) = X^\topβ$ )

Answer 1

1: In case the outputs we are trying to model are real numbers then $K = 1$. But if we output is a GPS coordinate, then we would possibly model the output space as $\mathbb{R}^2$. In that case $K = 2$. We would have

$(X_1, X_2, X_3) \cdot \left(\begin{array}{cc} \hat\beta^0_0 & \hat\beta^1_0 \\ \hat\beta^0_1 & \hat\beta^1_1 \\ \vdots & \vdots \\ \hat\beta^0_N & \hat\beta^1_N \\ \end{array}\right) = (\hat Y^0, \hat Y^1)$

where the $N \times 2$ matrix is the $p \times K$ matrix that is discussed in the book.

2: Consider the simple example where $p = 2$. We are given $\hat \beta$. We have $\hat Y = \hat \beta_0 + X_1 \hat \beta_1 + X_2 \hat \beta_2$. Then $(X, \hat Y)$ can be see as $\{ (x_1, x_2, \hat y) \} = \{ (x_1, x_2, \hat \beta_0 + x_1 \hat \beta_1 + x_2 \hat \beta_2) \} \subset \mathbb{R}^3$

3: That's a decent guess. A vector hyperplane $H$ of a vector space $V$ of dimension $n$ is a linear subspace that has dimension $n-1$. Because it is a subspace it must include the origin (since every vector space must contain the zero vector). An affine hyperplane is just a translation of a vector hyperplane: $a + V = \{\ a + v\ |\ v \in V\ \}$. To be concrete, any line in $\mathbb{R}^2$ is an affine hyperplane (since any line is a translation of some line through the origin) but only lines through the origin are vector hyperplanes.

4,5,6:

I genuinely think they made an error in the book and said the reverse of what they were trying to say. I think they mean to say that if the intercept is included in $\hat \beta$ that it's an affine subspace and that if the intercept is not included in $\hat \beta$ that it is a linear subspace. For example, in one dimension, $y = ax$ is a linear subspace (missing bias $b$) whereas $y = ax + b$ is an affine subspace since it doesn't contain the origin. But I'd like to be corrected if I interpreted this incorrectly.

7: With bias, they just mean the $\hat \beta_0$ term, the $y$-intercept. You shouldn't confuse it with other uses of the word bias.

8: Not a typo. They just mean $\beta$ in general. They're just talking about the specific shape of $\beta$, not any particular $\hat \beta$.