# In a Bayesian hierarchical model, if exchangeability doesn't hold, what exactly goes wrong?

In many textbooks, when a Bayesian model is presented, such as a classic Normal-Normal model, there is some sort of brief mention that the trials must be exchangeable. I am wondering why this is necessary and what goes wrong if exchangeability doesn't hold. Does anyone have any concise answers?

Exchangeability is not necessary. There are Bayesian models in which observations are not exchangeable. For example, models for time-series analysis and forecast in weather prediction or finance. Generally speaking, in such models more recent observations are considered to be more relevant for inference about future ones; a sort of "fading memory". Exchangeability therefore cannot be assumed for them. There is a huge variety of non-exchangeable models; see the references below.

Exchangeable models are often easier to deal with, but they may be inappropriate. In fact, rather than "wrong" vs "right", the question is whether exchangeability or other assumptions, like the "fading memory" mentioned above, are more appropriate or reasonable for the inferences you're making, or computationally easier. We must often find a balance between these two aspects.

There's no "right" or "wrong" because there's no experiment that can tell us whether an inference model is "correct". This is the fundamental issue of induction, about which many, many authors have written; I recommend the works of Hume, Johnson, Jeffreys, de Finetti, Jaynes cited below. We can only apply a particular way of doing induction, formalized as a statistical model, and then see if we're satisfied with it or not. And this satisfaction depends on many criteria, many of which subjective.

Texts like Bernardo & Smith: Bayesian Theory (Wiley 2000) focus more on exchangeability, but as they themselves remark (§ 1.4.1), their book is not meant to cover all kinds of inferences in Bayesian probability theory. Texts specifically focused on non-exchangeable models are for example:

• R. Prado, M. West: Time Series: Modeling, Computation, and Inference (CRC 2010) – this should be a good and recent starting point if you're already familiar with exchangeable models.

• A. Pole, M. West, J. Harrison: Applied Bayesian Forecasting and Time Series Analysis (Springer 1994)

• E. Greenberg: Introduction to Bayesian Econometrics (Cambridge 2008)

• A. Zellner: An Introduction to Bayesian Inference in Econometrics (Wiley 1996)

• G. L. Bretthorst: Bayesian Spectrum Analysis and Parameter Estimation (Springer 1988) http://bayes.wustl.edu/glb/bib.html

• G. E. Box, G. M. Jenkins, G. C. Reinsel, G. M. Ljung: Time Series Analysis: Forecasting and Control (Wiley 2016), especially ch. 7

• W. Palma: Long-Memory Time Series: Theory and Methods (Wiley 2007), especially ch. 8