I am trying to understand better the 'Learning' part in "Machine Learning', and pinpoint when it happens. Will use ** to highlight the key part.
This is what I think it is: It consists of tweaking the parameters of an algorithm. Let me apply it to regression specifically , since I understand it (more or less).
1) We are given data points $(x_1, y_1),.....,(x_n, y_n) $ which we separate into test data and and training data. We have a loss function f and a choice of threshold C . We want to satisfy the condition : || Residual ||< c , where Residual is the difference between the model and actual data ( i.e., Predicted - Observed).
2) If || Residual ||< c is satisfied, we exit.
3) If ||Residual ||$ \geq c$ ** . Then the regression model ( meaning here the choice of parameters ) is rejected and the computer ** Learns a better choice of parameters by gradient descent.
4) New parameters are used an we test again for whether || Residual ||
5) We go to 3)
Is this correct? If so, is the idea the same for different algorithms? If not, what I am missing? Thanks. the choice of parameters or coefficients
2) We use the test data to run our