'Learning' Part in Machine Learning? I am trying to understand better the 'Learning' part in "Machine Learning', and pinpoint when it happens. Will use ** to highlight the key part.
This is what I think it is: It consists of tweaking the parameters of an algorithm. Let me apply it to regression specifically , since I understand it (more or less).
1)  We are given data points $(x_1, y_1),.....,(x_n, y_n) $ which we separate into test data and and training data. We have a loss function f and a choice of threshold C . We want to satisfy the condition : || Residual ||< c , where  Residual is the difference between the model and actual data ( i.e., Predicted - Observed).
2) If || Residual ||< c is satisfied, we exit.
3) If ||Residual ||$ \geq c$ ** . Then the regression model ( meaning here
the choice of parameters ) is rejected and the computer ** Learns a better choice of parameters by gradient descent. 
4) New parameters are used an we test again for whether || Residual || 

5) We go to 3)
Is this correct? If so, is the idea the same for different algorithms? If not, what I am missing?
Thanks.
the choice of parameters or coefficients 
2) We use the test data to run our
 A: The "learning" part of machine learning is optimization. You've described a particular optimization problem.
A: The idea is this.  You have a set of data input pairs (x, y), or tuples, as I will call them (in the simplest case).  You want to (for example) predict y - or something about y - based on knowing the x it goes with.  You split it into a learning group and a test group.
Now you could be trying to 'teach' any number of approaches or algos.  It could be a neural network; it could be an SVM; it could also be old-fashioned regression.
However the mechanism works, you feed it x's and tell it if it's prediction is wrong (or how wrong it is).  The goal is for the algo to find a 'best' way to predict the y from any x.  Then you test in on the test data to make sure it really does predict (the big risk being overfitting of the training data).  #
Broadly speaking, there are two kinds of algos.  Ones like regression don't really, in my mind, involve 'machine learning'.  It's just applying standardized math to solve a known optimization problem (least squares).
But if you have a neural net it is a different beast, because the net does change and evolve to move toward it's best possible algo given the net design.
