1
$\begingroup$

What is the difference between parameter estimation which includes system identification and learning in machine learning perspective?

Let say the model is y= Ax. x is the input and y is the output. In estimation, I have seen parameters to be estimated, maybe in this case it is A and the samples are also estimated (unsure)

What is learning then?

$\endgroup$
2
  • 2
    $\begingroup$ Nothing. Same idea, different fields. $\endgroup$
    – Zoë Clark
    Commented Jul 30, 2014 at 1:45
  • 1
    $\begingroup$ Just a comment, this is a profound question and, while I agree with the accepted answer, I would like to see or hear some other perspectives from in-the-trenches data scientists. $\endgroup$
    – AdamO
    Commented Jul 27, 2021 at 17:27

2 Answers 2

4
+150
$\begingroup$

Basicaly same, but flavor of terms is a little different - by estimation people usually mean that you specify underlying distributions and then estimate their parameters. Learning may be distribution free - just optimizing some target function and it applies in situations with complex structured data when it not reasonable/possible to build a distributional model.

$\endgroup$
0
$\begingroup$

Estimators are rules for estimating a value based on observed data. We have estimators like GMM or MLE which estimate certain values, whether parametrically or not. These rules must satisfy certain properties like consistency, unbiasedness, efficiency, etc.

Learning is more general, and includes methods with or without estimators. K-nearest neighbors uses distance functions and not statistical estimation to classify or regress. Fine tuning the hyperparameter $k$ is also practically a matter of improving the error metric rather than any statistical estimation per se.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.