Context: I've used Principal Component Analysis a fair bit, toyed with Artificial Neural Nets, and only read about other machine learning techniques. I have some academic exposure to machine learning, but only chapters of books and individual lectures - never a complete course or text.
There seem to be "architectural" similarities among machine learning techniques. There always seems to be a training process step and a prediction process step. Moreover, there seems to be similarities in the data passed among those two processes:
- Tagged or not, there is always training data.
- The training process always outputs a trained model
- The training process and/or model are usually configurable somehow.
- The prediction process uses the trained model to predict something about its input data.
- The training data and the prediction input data are both constrained in similar ways (EX: both are strings, or both are 30 dimensional vectors, or both are 16x16 images, etc).
Do all machine learning techniques share this high level architecture?
Or, if this covers only a subset, what do people call that subset?
PCA and ANN, for example, both have a training step that takes training data to output a trained model. And both use that trained model to generate from data that hasn't yet been seen. Here's my attempt to hammer both of these techniques into the same diagram.
(Load the following images in a separate tab if you can't read them - they're designed for 150 pixels per inch.)
There seem to be a couple of established ways to do PCA (correlation method and covariance method), and neither one is very flexible or parametric. On the other hand, there seem to be N+1 interesting ways to configure or train neural nets, where N is the number of interesting ways in existence yesterday. Because of this, I imagine the difference gets pretty blurry between these two points:
- "two different ANN techniques"
- "two different flavors/configurations of one ANN techique"
I'm assuming there are other ML techniques which are similar in that respect. I draw attention to this technique-vs-flavor ambiguity because it's the most complicating factor I've found when I'm trying to accurately shove a particular ML technique into the boxes of the diagram.
I also recognize that the training process can include the prediction process, as in neural nets or evolutionary techniques. But, as far as I know, there is still a distinct trained model that can be used with a prediction process after training.