9
$\begingroup$

I know that Bayesian and frequentist approaches differ in their definition of probability.

Practically, in machine learning a model is a formula with tunable parameters.

Then the difference between Bayesian and frequentist is:

That the parameters are assumed to be fixed numbers in frequentist setting and the parameters have their own distributions in the Bayesian setting.

Am I missing anything here or anything is mis-interpreted?

I am not asking theoretical arguments, just what is the practical manifestation of frequentist vs Bayesian w.r.t. Machine learning models and their optimization/fitting.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Why do you say that they are different in their definition of probability ? Don't they use both the definition by Kolmogorov ? $\endgroup$
    – user83346
    Commented Nov 19, 2016 at 17:17
  • 1
    $\begingroup$ @fcop i read belief vs frequency.... $\endgroup$
    – Rafael
    Commented Nov 19, 2016 at 17:28
  • 1
    $\begingroup$ Take a look at related threads in the column on the right. Maybe you will find an answer to your question there. $\endgroup$
    – Richard Hardy
    Commented Nov 19, 2016 at 17:30
  • 1
    $\begingroup$ Then is it 'definition' or 'interpretation' ? $\endgroup$
    – user83346
    Commented Nov 19, 2016 at 17:51
  • 1
    $\begingroup$ Here you can read more about Bayesian way of looking at probability: stats.stackexchange.com/questions/173056/… $\endgroup$
    – Tim
    Commented Nov 19, 2016 at 18:41

1 Answer 1

Reset to default
10
$\begingroup$

Once you've fitted the model, it will be what it will be, so I think the difference is prior to that. That is, the models / parameters are fitted differently between the Bayesian and Frequentist approaches. More specifically, the fitted Bayesian parameters will incorporate additional information outside of what is in the data. If you know something about what the parameters are likely to be (and you aren't wrong), that could boost the model's performance. Even if you use an 'uninformative' prior, you will typically find the fitted Bayesian parameters will be shrunk to some degree towards $0$ relative to the fitted Frequentist parameters.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.