Linear decision function (classification) Although I know some basics of linear classification, I do have some questions about the formalism. In our script, a binary linear classifier $F$ is defined as follows:
$$
F(x) = \textrm{sign}\left(\langle w, x \rangle + b\right) \in \{-1, 1\}
$$
where
$$
\textrm{sign}(z) = \begin{cases}1 \hspace{8.5mm} \textrm{ if } z \geq 0 \\ -1 \hspace{5mm} \textrm{ o.w.} \end{cases}
$$
for the weight vector $w$, the intercept $b$, and $\langle.,.\rangle$ being the dot product.
My questions:
(1) What exactly do we need the bias $b$ here for? As far as I understand, we multiply the weight vector w with the feature vector x, obtaining a scalar, which we then compare to a threshold. Hence the only use I see here in the including an intercept is to set that threshold to zero (as is actually done above) - is my understanding correct? In fact, if we always compare a scalar to a threshold, is the entire problem not reducible to the real line?
(2) Parameterization: It is stated that due to the fact that we can normalize the weight vector to have norm 1, the function has only $d$ parameters (in contrast to the weight vector not being normalized, where we would have $d+1$ parameters). Is this because the last entry of the weight vector would be determined through the normalization?
 A: (1) Think about what would happen if we didn't have $b$: our classifier would be $F(x) = \textrm{sign}(w^T x)$ which always goes through the origin. This will severely limit the patterns that we can separate.
In the example below, we can see that the data are linearly separable, but any line going through the origin will do a very bad job. It seems absurd to restrict ourselves to such lines, so what we do is we translate all of our points over by subtracting $b$, and then a line through the origin can indeed separate them.

(2) you are correct. Let's say we've got a vector $x := (x_1, x_2, x_3, x_4) \in \mathbb R^4$. Just based on that, the values in $x$ are unconstrained, i.e. we have 4 parameters that we are free to vary. But if now we know that $x^T x = 1$, then this means that $x_4^2 = 1 - x_1^2 - x_2^2 - x_3^2$ so we now only have three quantites which can be varied since the fourth is determined by the other three.
Update
You asked about whether or not we can get the same decision boundary with or without an intercept by modifying $w$. I'll answer that in detail here. One important thing to remember about this is that we need the values to match up for all $n$ observations, not just for a single observation.
So let $X \in \mathbb R^{n \times p}$ be our data matrix and let $(w, b) \in \mathbb R^{p+1}$ be the vector normal to our hyperplane and the intercept (i.e. the usual $w$ and $b$). I'll let $\vec b$ denote the vector in $\mathbb R^n$ of all $b$.
We want to find a vector $w^* \in \mathbb R^p$ such that $Xw + \vec b = Xw^*$. This is equivalent to solving the system
$$
X(w^* - w) = \vec b.
$$
Now we can see that if $X$ has a constant column (or just if it has a constant vector in its span) there exists a vector $v$ such that $Xv = \vec b$, i.e. $\vec b$ is in its column space and we can indeed just work with $\langle w^*, x \rangle$.
But in general we often have $n >> p$ so the span of the columns of $X$ is far less than $R^n$ and there's no reason why $\vec b$ should be in there. That means that in general we cannot exactly mimic the behavior of $\langle w, x \rangle + b$ with some other $\langle w^*, x \rangle$ unless we specifically modify our $X$ to guarantee that this is possible, such as by appending a constant column.
