# Bayesian inference on the maximum of a function

Suppose,

$$Y\mid X, \beta_1, \beta_2 \sim N(\beta_1X + \beta_2X^2, \sigma^2)$$

With $$\sigma^2$$ known and priors on $$\beta_1$$ and $$\beta_2$$ that are a mixture of 50% point mass on zero, and 50% a diffuse normal centered at zero and large variance. This prior comes from real physical understanding of the problem, and we do not want to change the prior, that's the whole purpose of the question.

I want to make inference regarding the value of $$X^*$$ that maximizes the expected value of $$Y$$.

We know that $$X^*$$ is just a (sometimes set-valued) function (a transformation) of the parameters $$\beta_1, \beta_2$$. For instance, if the function is concave, the maximum is a point, and given by,

$$X^* = -\frac{\beta_1}{2\beta_2}$$

If the function is constant, that is, $$\beta_1 = \beta_2 =0$$, then the maximum is a set, $$X^* = (-\infty, +\infty)$$, and so on.

What is the posterior distribution of $$X^*$$? Or how can we get the posterior distribution of $$X^*$$ from the posterior of $$\beta_1$$ and $$\beta_2$$? Or, an even simpler question, what is the implied prior on $$X^*$$? And finally, how would you implement this in standard software such as Stan? I failed miserably on this...

• The posterior is a Cauchy. Feb 1 '20 at 7:17
• @Xi'an conditional on the function being concave, yes it is a cauchy. But what about the 50% point mass on zero? When $\beta_1 = \beta_2 = 0$, the "maximum" is the whole real line, for instance.
– user272422
Mar 13 '20 at 1:37
• This is related to response surface methods. Search this site, and google.com/… Mar 13 '20 at 12:33
• Just to be clear, you are seeking to maximise $\mathbb{E}(Y|X = x)$ with respect to $x$ right? (I.e., you want to marginalise out the beta values and then maximise the remaining conditional expectation?)
– Ben
May 19 '20 at 2:47
• We are familiar with calculus.
– Ben
May 19 '20 at 15:48

This is a Bayesian regression problem where you have imposed a prior on the regression coefficients and you are seeking to maximise the conditional expectation function $$\mathbb{E}(Y|X=x)$$. You are using a polynomial regression model conditional on the parameters $$\beta_1$$ and $$\beta_2$$, with the true regression function:

$$\mathbb{E}(Y|X=x, \beta_1, \beta_2) = \beta_1 x + \beta_2 x^2.$$

You can find the conditional expectation function explicitly using the law of iterated expectation.

\begin{aligned} \mathbb{E}(Y|X=x) &= \mathbb{E} \Big( \mathbb{E}(Y|X=x, \beta_1, \beta_2) \Big) \\[6pt] &= \mathbb{E} ( \beta_1 x + \beta_2 x^2 ) \\[6pt] &= x \cdot \mathbb{E}(\beta_1) + x^2 \cdot \mathbb{E}(\beta_2) \\[6pt] &= x \cdot 0 + x^2 \cdot 0 \\[6pt] &= 0. \\[6pt] \end{aligned}

Since both of your priors have zero expectation, we can see that $$\mathbb{E}(Y|X=x) = 0$$ for all $$x \in \mathbb{R}$$. This means that any value maximises the function.

• Thanks, but I'm not sure this answers the question though, namely: what is the implied prior on $X^*$? The whole real line $\mathbb{R}$ is a solution only certain percent of the time, say 25%. Do we treat the whole set of real numbers $\mathbb{R}$ as an object itself, and assign to it the 25% probability? Or, do we distribute 25% to among the solutions in $\mathbb{R}$? How do I write down this implied prior on $X^*$?
– user272422
May 24 '20 at 3:12
• Just to be clear, $X^*$ is a parameter, it is a functional of the joint probability distribution. A prior on $\beta$ induces a prior on $X^*$. I want to know how to express the prior on $X^*$ induced by the spike and slab prior on $\beta$ (and also the posterior, but we need to know how to do the prior first).
– user272422
May 24 '20 at 3:16
• The value $X^*$ is not well-defined, since it is the maximising values of a constant function.
– Ben
May 24 '20 at 3:56
• When the function is a constant, it is well defined, but it is a set (the whole real line) instead of a number.
– user272422
May 24 '20 at 4:28