Why are they called "machines"? Is there an origin to the word "machine" used in this context? (Like the name "linear programming" can be confusing but we know why it is called "programming.")
Possibly because some of the earliest machine learning algorithms were implemented as actual physical machines. From Wikipedia:
The perceptron was intended to be a machine, rather than a program, and while its first implementation was in software for the IBM 704, it was subsequently implemented in custom-built hardware as the "Mark 1 perceptron".
(Picture from Cornell Library via Wikipedia)
ADALINE (Adaptive Lineardi Neuron or later Adaptive Linear Element) is an early single-layer artificial neural network and the name of the physical device that implemented this network.
A source for more information about these machines is the book Talking Nets.
I think this article sums it up.
Basically, machine comes from machine learning, a term that was coined in 1959 by Arthur Samuel, way before the final developments that led to the soft marging kernel SVM in the 90's and the Boltzmann Machines in the 80's. Vapnik and Lerner called their algorithm the Generalized Portrait algorithm back in 1963, see more here.
A machine in this context would be the output function, or as the article puts for historical reasons, the hypothesis, built from the learned parameters.
Meriam-Webster defines the word as "a literary device or contrivance introduced for dramatic effect." It certainly does the job of increasing the dramatic effect. "Support vector algorithm/approach/equation/function/..." just doesn't sound as good as "support vector MACHINE!"
I would also suggest "kernel density machine" and "maximum likelihood machine." Actually, I'm going to call all my algorithms "machines" from now on.