5
$\begingroup$

currently I am comparing different combination methods:

  1. Equally Weight Averaging -> deterministic
  2. Ordinary Least Squares Averaging -> frequentist?
  3. Bayesian Information Criterion Averaging -> both, bayesian and frequentist?
  4. Bayesian Model Averaging -> bayesian

For the two models in the middle I am not sure If my classification is correct?

Improve this question
$\endgroup$
2
  • $\begingroup$ Ordinary Least Squares can be a Bayesian method and indeed it was discovered before anything other than Bayesian methods existed. The BIC is used by both, but it is primarily a Frequentist method because it is really a cost function applied to the posterior. Equally weighted averaging can be either Frequentist or Bayesian, as there are circumstances where a Bayesian would do that. The sample average, and hence the OLS estimators, are often sufficient statistics and so can be used by either. $\endgroup$
    – Dave Harris
    Dec 5, 2016 at 16:17
  • 2
    $\begingroup$ The question seems motivated by a conceptual confusion, but that shouldn't be a problem--if an OP already understood everything, there's no need to ask. In light of the well up-voted answer, I don't think this question is too unclear. I'm voting to leave open. $\endgroup$
    – gung - Reinstate Monica
    Dec 5, 2016 at 17:56

1 Answer 1

Reset to default
17
$\begingroup$

You're making a familiar category error here.

The methods you are talking about all correspond to some logical or algebraic structure or other, which is 'just math'. Similarly, they each have a particular implementation, which we might describe as 'just programming'. Neither math nor programming are well described as Frequentist, Bayesian, or anything like that.

Put another way, the same mathematical structure can be Frequentist, Bayesian, some third thing, or a mix depending on how its elements are used for inference (which is something you do, not something the model has or does).

Two examples

OLS is an algorithm (minimize the sum of squared errors) and can be motivated as a Frequentist method when it is chosen for the behavior of its output in repeated samples, or a Bayesian tool for getting a key parameter of the posterior distribution under certain assumptions about the prior distribution of parameters, or as neither when it is motivated as an interesting application of singular value decomposition, or some other linear algebraic tool.

Brown, Cai and Dasgupta, 2001 show that a Jeffrey's prior on a binomial proportion - something at least notionally Bayesian - behaves very well in repeated samples, and can be justified in a Frequentist way (Section 4.3), that is, in a way that makes no mention of beliefs about the value of the parameter.

An analogy

Think about dessert wines. A dessert wine is a wine that typically drunk with dessert, not a wine made in a special way or from a special kind of grape. It's true that dessert wines tend to have some characteristic properties, e.g. they're sweeter, but those features are not what makes them dessert wines; that's the dessert.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ Except for the dessert wine analogy, I completely concur with your answer! $\endgroup$
    – Xi'an
    Dec 5, 2016 at 21:29
  • $\begingroup$ Quoi? Sauternes à l'aube, monsieur! $\endgroup$
    – conjugateprior
    Dec 6, 2016 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.