Why is the bayesian information criterion called that way? The word "Bayes" suggests that we are updating a distribution using data, to get a posterior distribution. 
The fact that the Bayesian information criterion (BIC) is used to select a model from a set of models, suggests that it is called BIC because we are selecting the model with the highest posterior, or something like that.
However, it's not clear to me from the definition of BIC that that's the case. Is it? What is the reason that the Bayesian information criterion is called "Bayesian"?
 A: BIC, sometimes called the Schwarz information criterion (SIC) was introduced by Gideon Schwartz in 1975. Here is that paper. It's not very long.
Both AIC and BIC address the model evaluation problem where more parameters lead to increased likelihood. To resolve this they penalize additional parameters. Schwartz gave a bayesian argument. His central thesis was, "In a model of given dimension, maximum likelihood estimators can be obtained as large-sample limits of the Bayes estimators for arbitrary nowhere vanishing a priori distributions." 
So even though the formula for BIC has nothing Bayesian about it, the original paper is chock full of Bayesian explanations for the procedure since it was derived from a special "large-sample" case of Bayes Theorem.
A: One of my favourite simple interpretations and derivation of the BIC is in a lecture by Prof. Michael Jordan. This connects the Laplace approximation of the marginal likelihood (normalising constant in the Bayes Theorem, which is used for model comparison) with the BIC.
https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture16.pdf
Here, he derives the BIC using a Laplace approximation and some intuitive asymptotic arguments.
Laplace approximation:

Then,

Finally,

