You should be using dummies for all of these approaches for exactly the reason you specified: the numeric value by which the categories happen to be encoded doesn't capture similarity (e.g. 4 isn't closer to 3 than it is to 1).
If you have a model with linear components of the form $\beta_i x_i$ (all three of the methods you mentioned do, even kernel SVM), then by not using dummy variables you are stating by design that 4 is closer to 3 than it is to 1. Suppose you have 3 instances with values 1, 3 and 4, then their contribution to (parts of) the prediction is $$\begin{align} x_i = 1 \rightarrow\ &\beta_i, \\ x_i = 3 \rightarrow\ &3\beta_i, \\ x_i = 4 \rightarrow\ &4\beta_i, \end{align}$$ and hence the contribution of $x_i=4$ appears more similar to $x_i=3$ than to $x_i=1$, even though they aren't. With dummies you completely avoid this issue.
Techniques like decision trees don't require such encoding, because they generate splits rather than use the numeric values directly (the splits will land somewhere between the categories and thus effectively separate them).