Confused on Bayesian Decision Theory

Question

I am trying to understand what is the right way to pick up an "action", as it is called in Murphy, Machine Learning a Probabilistic Perspective, in the 'chatper 'Bayesian decision theory'.

I'll make the question more clear as in the previous post it wasn't. Suppose our data comes froma a linear process $Y = \beta X + \epsilon$. Theoretically, given a set of training data $D = (\mathbb{Y}, \mathbb{X})$ one would like to minimize the generalization error $E_{Y|X,\theta^*, D}[\mathcal{L}(Y, g(X, D))|D]$ where $\mathcal{L}$ is a loss function, $g$ is a prediction model we trained on $D$, that is, once given a new input it outputs a decision $\hat{Y}$, and the expectation is taken with respect to the true, unknown, data generating process, that is simply $Y = \beta X + \epsilon$ for unknown $\beta$ for the present example. The only way we have to estimate the generalization error is to use a separated test set, but if we use it to select a prediction model this of course no longer is a test set. So in frequentist theory, given that the objective is the generalization error one search for strategy to evaluate it, its mean over the training sets, some other proxy of it or uses some theoretical selection creiteria all based on the idea that you training is one out of the possibles under the true DGP.

Now, what is not clear to me is how you should reason in bayesian framework, what is your objective. So, for what I understand, even in bayesian framework to assess the generalization capacity of my model I need a test set. If I did it with respect to my posterior I would end up be fine with something that is actually not fine I suppose. So, even in a bayesian framework my ideal objective would be the generalization error previously defined, where the average is taken with respect to the true DGP. So, even in bayesian framework we can't directly minimize it, as we can't use the test data to pick up a model.

So, question 1, is it correct that what we do, in being bayesian, is to try to approximate this generlization error by taking its mean over the posterior predictive?

So, instead of:

$ err(g) = E_{Y|X,\theta^*}[\mathcal{L}(Y, g(X, D))] =\int \mathcal{L}(y, g(X, D))p(Y|X, \theta^*)\mathrm{d}y = \int \mathcal{L}(y, \beta_{\lambda}(D)X) p(Y|X, \theta^*)\mathrm{d}y $

that is the generalization error of interest, where in the last equality I explicitely write the forecast relative to a given estimate of the parametrs under a ridge penalty at a specific value $\lambda$, we evaluate:

$ \int \mathcal{L}(y, g(X, D))p(y|X, g)p(g|D)\mathrm{d}y\mathrm{d}g = \int \mathcal{L}(y, \beta_{\lambda}X)p(y|X, \beta)p(\beta|\lambda, D) p(\lambda |D)\mathrm{d}y\mathrm{d}\beta\mathrm{d}\lambda $

And then another point of confusion. Assume that we have a ridge model prior and be $\lambda$ the hyperparameter of such a prior. And assume we are using a quadratic loss, using absolute value does not cheange my point.

How do I build my forecast on a new unseen data X?

Option 1: Be

$ p(Y|X, D) = \int p(Y|X, \beta)p(\beta|D, \lambda) p(\lambda | D) \mathrm{d}\beta\mathrm{d}\lambda $

my posterior predictive. If your minimization problem is:

$ a^*(X) = \mathrm{argmin}_{a: \mathcal{D}_X \to \mathcal{D}_Y}\int\mathcal{L}(y, a(X))P(y|X, D) \mathrm{d}y $

then your forecast $a(X)$, for a quadratic loss, would be the mean of the posterior.

Option 2:

$ a^*(X) = \mathrm{argmin}_{a}\int\mathcal{L}(y, a(X, \beta, \lambda))p(Y|X, \beta)p(\beta|D, \lambda) p(\lambda | D) \mathrm{d}\theta\mathrm{d}\lambda \mathrm{d}y $

In this second case, it is not clear to me how should I pick up a but for re-evaluating the integral at each new point.

Answer 1

I think you are confused by how the various elements integrate with one-another. Let's start with the basic definitions and then build-up to how they interact together

Optimal decision making under randomness

We want to find the "best action" under "imperfect knowledge":

• To model imperfect knowledge, we will use probability theory: let $Y$ be some random variable with distribution $F$

• to model "best action" we will choose a loss function which maps:

  • an action $a$
  • a value $y$

  to a concrete loss value $l(y, a)$

The optimal decision for $F$ can then be defined as the one that minimizes the expected loss:

$$ a \rightarrow \mathbb E [ l(Y, a) ] $$

Prediction in Bayesian inference

When applying Bayes' law to a model with:

• some latent parameters $\theta$
• some observed data $x$
• some new situation to predict $y$

we can compute:

• the posterior distribution of $\theta$ conditional on $x$:

  $$ p(\theta | x) $$

• the joint posterior distribution of $y, \theta$

  $$ p(\theta, y | x) = p(\theta | x) p(y | \theta) $$

  NB: in full generality, we should have $p(y | \theta, x)$. I've assumed that $x,y$ are independent conditional on $\theta$ which is commonly true.

• the marginal posterior of $y$:

  $$ p(y | x) = \int p(\theta | x) p(y | \theta) d\theta $$

  this is the quantity we would need for forecasting and optimal decision making

Notes:

• notation is slightly different than your example
• exact inference could be replaced by approximate inference
• you can view this a combining model-uncertainty and datapoint-uncertainty (epistemic and aleatoric uncertainty, if you're being fancy)

Generalization error

Given a loss $L(y,a)$, we would like to evaluate some machine-learning pipeline:

• the algorithm needs to be written so that its output is an action $a$
• we can compute the expected loss under some model of the data (NB: this is going to lead to random actions: the input of the ML algorithm will also be random)
• we can compute a concrete loss for some test dataset

Note that, if I'm evaluating a Bayesian method:

• to choose the optimal action, the Bayesian method will need some notion of loss

• this can be different than the loss I'm using for evaluation

• there are two radically-different expected values:

  • one for computing the optimal action; that's an expected value under the posterior
  • one for computing the generalization loss; that's an expected value under some model of the test data

I hope this helps.

Edit: these are typical difficulties

I want to add a further note. These difficulties are typical when one learns Bayesian inference (and other statistical frameworks).

• the theory calls upon many different probability models which are intertwined.

  • There is the Bayesian model, which we invert to construct our predictions. It contains both marginal, joint and conditional probabilities.
  • There is the test dataset.
  • There is, perhaps, an underlying test model of which the test data is a realization.
  • etc

• the notation and exposition might not make it obvious how these models are related to one-another and what role they play

• it is thus unsurprising that it takes a while to master these difficulties. I think everybody has been there.