It is easier to optimise a SVM without an offset term $b$ since we get rid of one of the linear constraints in the dual formulation. If an offset is still needed, it has been suggested that the feature vector $x\in\mathbf{R}^n$ is augmented by a constant term. Let's call this term $d$, then $x' = [x; d]\in\mathbf{R}^{n+1}$.
Suykens et al 2014 (page 9 in Regularization, Optimization, Kernels, and Support Vector Machines) write that
As a side-effect, the offset $d^2$ is then also regularized in the new term $||w||^2$. Nevertheless, if desired, the effect of this additional regularization can be made arbitrarily weak by re-scaling the fixed additional feature value from one to a larger value.
A similar argument was made here by Sobi in a different thread
However, if we increase $d$ (to make its effect on regularization smaller) we make the effect on the kernel matrix larger. E.g. for a linear kernel and another augmented sample $y' = [y; d]$ we get
$$ K(x',y') = x'^\top y' = x^\top y + d^2 = K(x,y) + d^2 $$
Is this effect on the kernel matrix problematic? Should we hence not make $d$ too large?
