Return to Answer

It would indeed make sense that a "good" prior distribution gives positive probability or positive density value to the "true" parameter $\theta_0$, but from a purely decisional perspective this does not have to be the case. A simple counter-example to this "intuition" that$$\pi(\theta_0)>0$$should be necessary, when $\pi(\cdot)$ is the prior density and $\theta_0$ is the "true" value of the parameter, is the brilliant minimaxity result of Casella and Strawderman (1981): when estimating a Normal mean $\mu$ based on a single observation $x\sim{\cal N}(\mu,1)$ with the additional constraint that $|\mu|<\rho$, if $\rho$ is small enough, $\rho\le 1.0567$ specifically, the minimax estimator corresponds to a (least favourable) uniform prior on $\{-\rho,\rho\}$, meaning that $\pi$ gives equal weight to $-\rho$ and $\rho$ (and none to any other value of the mean $\mu$) $$\pi(\theta)=\frac{1}{2}\delta_{-\rho}(\theta)+ \frac{1}{2}\delta_{\rho}(\theta)$$ When $\rho$ increases the least favourable prior sees its support growing, but remaining a finite set of possible values. However the posterior expectation, $\mathbb{E}[\mu|x]$, can take any value on $(-\rho,\rho)$.

The core of the discussion (see comments) may be that, were the Bayes estimator to be constrained to be a point in the support of $\pi(\cdot)$, its properties would be quite different.

Similarly, when considering admissible estimators, Bayes estimators associated with a proper prior on a compact set are usually admissible, although they have a restricted support.

In both cases, the frequentist notion (minimaxity or admissibility) is defined over the possible range of parameters rather that at the "true" value of the parameter (which brings an answer to Question 4.) Furthermore, as pointed out in the above example, when the Bayes estimator is defined by a formal expression such as the posterior mean $$\hat{\theta}^\pi(x)=\int_\Theta \theta\pi(\theta|x)\text{d}\theta$$ for the quadratic (or $L_2$) loss, this estimator may take values outside the support of $\pi$ in cases this support is not convex.

As an aside, when reading

for the true θ to have generated the data (i.e. "exist"), θ must be a possible variate under π, e.g. have non-zero probability, non-zero density

I consider it a misrepresentation of the meaning of a prior. The prior distribution is not supposed to stand for an actual physical (or real) mechanism that saw a parameter value $\theta_0$ generated from $\pi$ followed by an observation $x$ generated from $f(x|\theta_0)$. The prior is a reference measure on the parameter space that incorporates prior information and subjective beliefs about the parameter and that is by no means unique. A Bayesian analysis is always relative to the prior chosen to conduct this Bayesian analysis. Hence, there is not an absolute necessity for the true parameter to belong to the support of $\pi$. Obviously, when this support is a compact connected set, ${\mathscr A}$, any value of the parameter outside the set ${\mathscr A}$ cannot be consistently estimated by the posterior mean $\hat{\theta}^\pi$ but this does not even prevent the estimator to be admissible.

Very nice question! It would indeed make sense that a "good" prior distribution gives positive probability or positive density value to the "true" parameter $\theta_0$, but from a purely decisional perspective this does not have to be the case. A simple counter-example to this "intuition" that$$\pi(\theta_0)>0$$should be necessary, when $\pi(\cdot)$ is the prior density and $\theta_0$ is the "true" value of the parameter, is the brilliant minimaxity result of Casella and Strawderman (1981): when estimating a Normal mean $\mu$ based on a single observation $x\sim{\cal N}(\mu,1)$ with the additional constraint that $|\mu|<\rho$, if $\rho$ is small enough, $\rho\le 1.0567$ specifically, the minimax estimator corresponds to a (least favourable) uniform prior on $\{-\rho,\rho\}$, meaning that $\pi$ gives equal weight to $-\rho$ and $\rho$ (and none to any other value of the mean $\mu$) $$\pi(\theta)=\frac{1}{2}\delta_{-\rho}(\theta)+ \frac{1}{2}\delta_{\rho}(\theta)$$ When $\rho$ increases the least favourable prior sees its support growing, but remaining a finite set of possible values. However the posterior expectation, $\mathbb{E}[\mu|x]$, can take any value on $(-\rho,\rho)$.

The core of the discussion (see comments) may be that, were the Bayes estimator to be constrained to be a point in the support of $\pi(\cdot)$, its properties would be quite different.

Similarly, when considering admissible estimators, Bayes estimators associated with a proper prior on a compact set are usually admissible, although they have a restricted support.

In both cases, the frequentist notion (minimaxity or admissibility) is defined over the possible range of parameters rather that at the "true" value of the parameter (which brings an answer to Question 4.) For instance, looking at the posterior risk $$\int_\Theta L(\theta,\delta) \pi(\theta|x)\text{d}\theta$$ or at the Bayes risk $$\int_{\cal X}\int_\Theta L(\theta,\delta) \pi(\theta)f(x|\theta)\text{d}\theta\text{d}x$$ does not involve the true value $\theta_0$.

Furthermore, as pointed out in the above example, when the Bayes estimator is defined by a formal expression such as the posterior mean $$\hat{\theta}^\pi(x)=\int_\Theta \theta\pi(\theta|x)\text{d}\theta$$ for the quadratic (or $L_2$) loss, this estimator may take values outside the support of $\pi$ in cases this support is not convex.

As an aside, when reading

for the true θ to have generated the data (i.e. "exist"), θ must be a possible variate under π, e.g. have non-zero probability, non-zero density

I consider it a misrepresentation of the meaning of a prior. The prior distribution is not supposed to stand for an actual physical (or real) mechanism that saw a parameter value $\theta_0$ generated from $\pi$ followed by an observation $x$ generated from $f(x|\theta_0)$. The prior is a reference measure on the parameter space that incorporates prior information and subjective beliefs about the parameter and that is by no means unique. A Bayesian analysis is always relative to the prior chosen to conduct this Bayesian analysis. Hence, there is not an absolute necessity for the true parameter to belong to the support of $\pi$. Obviously, when this support is a compact connected set, ${\mathscr A}$, any value of the parameter outside the set ${\mathscr A}$ cannot be consistently estimated by the posterior mean $\hat{\theta}^\pi$ but this does not even prevent the estimator to be admissible.

A simple counter-example to this "intuition" that$$\pi(\theta_0)>0$$should be necessary, when $\pi(\cdot)$ is the prior density and $\theta_0$ is the "true" value of the parameter, is the brilliant minimaxity result of Casella and Strawderman (1981): when estimating a Normal mean $\mu$ based on a single observation $x\sim{\cal N}(\mu,1)$ with the additional constraint that $|\mu|<\rho$, if $\rho$ is small enough, $\rho\le 1.0567$ specifically, the minimax estimator corresponds to a (least favourable) uniform prior on $\{-\rho,\rho\}$, meaning that $\pi$ gives equal weight to $-\rho$ and $\rho$ (and none to any other value of the mean $\mu$) $$\pi(\theta)=\frac{1}{2}\delta_{-\rho}(\theta)+ \frac{1}{2}\delta_{\rho}(\theta)$$ When $\rho$ increases the least favourable prior sees its support growing, but remaining a finite set of possible values. However the posterior expectation, $\mathbb{E}[\mu|x]$, can take any value on $(-\rho,\rho)$.

The core of the discussion (see comments) may be that, were the Bayes estimator to be constrained to be a point in the support of $\pi(\cdot)$, its properties would be quite different.

Similarly, when considering admissible estimators, Bayes estimators associated with a proper prior on a compact set are usually admissible, although they have a restricted support.

In both cases, the frequentist notion (minimaxity or admissibility) is defined over the possible range of parameters rather that at the "true" value of the parameter (which brings an answer to Question 4.) Furthermore, as pointed out in the above example, when the Bayes estimator is defined by a formal expression such as the posterior mean $$\hat{\theta}^\pi(x)=\int_\Theta \theta\pi(\theta|x)\text{d}\theta$$ for the quadratic (or $L_2$) loss, this estimator may take values outside the support of $\pi$ in cases this support is not convex.

As an aside, when reading

for the true θ to have generated the data (i.e. "exist"), θ must be a possible variate under π, e.g. have non-zero probability, non-zero density

I consider it a misrepresentation of the meaning of a prior. The prior distribution is not supposed to stand for an actual physical (or real) mechanism that saw a parameter value $\theta_0$ generated from $\pi$ followed by an observation $x$ generated from $f(x|\theta_0)$. The prior is a reference measure on the parameter space that incorporates prior information and subjective beliefs about the parameter and that is by no means unique. A Bayesian analysis is always relative to the prior chosen to conduct this Bayesian analysis. Hence, there is not an absolute necessity for the true parameter to belong to the support of $\pi$. Obviously, when this support is a compact connected set, ${\mathscr A}$, any value of the parameter outside the set ${\mathscr A}$ cannot be consistently estimated by the posterior mean $\hat{\theta}^\pi$ but this does not even prevent the estimator to be admissible.

It would indeed make sense that a "good" prior distribution gives positive probability or positive density value to the "true" parameter $\theta_0$, but from a purely decisional perspective this does not have to be the case. A simple counter-example to this "intuition" that$$\pi(\theta_0)>0$$should be necessary, when $\pi(\cdot)$ is the prior density and $\theta_0$ is the "true" value of the parameter, is the brilliant minimaxity result of Casella and Strawderman (1981): when estimating a Normal mean $\mu$ based on a single observation $x\sim{\cal N}(\mu,1)$ with the additional constraint that $|\mu|<\rho$, if $\rho$ is small enough, $\rho\le 1.0567$ specifically, the minimax estimator corresponds to a (least favourable) uniform prior on $\{-\rho,\rho\}$, meaning that $\pi$ gives equal weight to $-\rho$ and $\rho$ (and none to any other value of the mean $\mu$) $$\pi(\theta)=\frac{1}{2}\delta_{-\rho}(\theta)+ \frac{1}{2}\delta_{\rho}(\theta)$$ When $\rho$ increases the least favourable prior sees its support growing, but remaining a finite set of possible values. However the posterior expectation, $\mathbb{E}[\mu|x]$, can take any value on $(-\rho,\rho)$.

The core of the discussion (see comments) may be that, were the Bayes estimator to be constrained to be a point in the support of $\pi(\cdot)$, its properties would be quite different.

Similarly, when considering admissible estimators, Bayes estimators associated with a proper prior on a compact set are usually admissible, although they have a restricted support.

In both cases, the frequentist notion (minimaxity or admissibility) is defined over the possible range of parameters rather that at the "true" value of the parameter (which brings an answer to Question 4.) Furthermore, as pointed out in the above example, when the Bayes estimator is defined by a formal expression such as the posterior mean $$\hat{\theta}^\pi(x)=\int_\Theta \theta\pi(\theta|x)\text{d}\theta$$ for the quadratic (or $L_2$) loss, this estimator may take values outside the support of $\pi$ in cases this support is not convex.

As an aside, when reading

for the true θ to have generated the data (i.e. "exist"), θ must be a possible variate under π, e.g. have non-zero probability, non-zero density

I consider it a misrepresentation of the meaning of a prior. The prior distribution is not supposed to stand for an actual physical (or real) mechanism that saw a parameter value $\theta_0$ generated from $\pi$ followed by an observation $x$ generated from $f(x|\theta_0)$. The prior is a reference measure on the parameter space that incorporates prior information and subjective beliefs about the parameter and that is by no means unique. A Bayesian analysis is always relative to the prior chosen to conduct this Bayesian analysis. Hence, there is not an absolute necessity for the true parameter to belong to the support of $\pi$. Obviously, when this support is a compact connected set, ${\mathscr A}$, any value of the parameter outside the set ${\mathscr A}$ cannot be consistently estimated by the posterior mean $\hat{\theta}^\pi$ but this does not even prevent the estimator to be admissible.

A simple counter-example to this "intuition" that$$\pi(\theta_0)>0$$should be necessary, when $\pi(\cdot)$ is the prior density and $\theta_0$ is the "true" value of the parameter, is the brilliant minimaxity result of Casella and Strawderman (1981): when estimating a Normal mean $\mu$ based on a single observation $x\sim{\cal N}(\mu,1)$ with the additional constraint that $|\mu|<\rho$, if $\rho$ is small enough, $\rho\le 1.0567$ specifically, the minimax estimator corresponds to a (least favourable) uniform prior on $\{-\rho,\rho\}$, meaning that $\pi$ gives equal weight to $-\rho$ and $\rho$ (and none to any other value of the mean $\mu$) $$\pi(\theta)=\frac{1}{2}\delta_{-\rho}(\theta)+ \frac{1}{2}\delta_{\rho}(\theta)$$

Similarly, when considering admissible estimators, Bayes estimators associated with a proper prior on a compact set are usually admissible, although they have a restricted support.

In both cases, the frequentist notion (minimaxity, admissibility) is defined over the possible range of parameters rather that at the "true" value of the parameter (which brings an answer to Question 4.) Furthermore, when the Bayes estimator is defined by a formal expression such as the posterior mean $$\hat{\theta}^\pi(x)=\int_\Theta \theta\pi(\theta|x)\text{d}\theta$$ for the quadratic (or $L_2$) loss, this estimator may take values outside the support of $\pi$ in cases this support is not convex.

As an aside, when reading

for the true θ to have generated the data (i.e. "exist"), θ must be a possible variate under π, e.g. have non-zero probability, non-zero density

I consider it a misrepresentation of the meaning of a prior. The prior distribution is not supposed to stand for an actual physical (or real) mechanism that saw a parameter value $\theta_0$ generated from $\pi$ followed by an observation $x$ generated from $f(x|\theta_0)$. The prior is a reference measure on the parameter space that incorporates prior information and subjective beliefs about the parameter and that is by no means unique. A Bayesian analysis is always relative to the prior chosen to conduct this Bayesian analysis. Hence, there is not an absolute necessity for the true parameter to belong to the support of $\pi$. Obviously, when this support is a compact connected set, ${\mathscr A}$, any value of the parameter outside the set ${\mathscr A}$ cannot be consistently estimated by the posterior mean $\hat{\theta}^\pi$ but this does not even prevent the estimator to be admissible.

A simple counter-example to this "intuition" that$$\pi(\theta_0)>0$$should be necessary, when $\pi(\cdot)$ is the prior density and $\theta_0$ is the "true" value of the parameter, is the brilliant minimaxity result of Casella and Strawderman (1981): when estimating a Normal mean $\mu$ based on a single observation $x\sim{\cal N}(\mu,1)$ with the additional constraint that $|\mu|<\rho$, if $\rho$ is small enough, $\rho\le 1.0567$ specifically, the minimax estimator corresponds to a (least favourable) uniform prior on $\{-\rho,\rho\}$, meaning that $\pi$ gives equal weight to $-\rho$ and $\rho$ (and none to any other value of the mean $\mu$) $$\pi(\theta)=\frac{1}{2}\delta_{-\rho}(\theta)+ \frac{1}{2}\delta_{\rho}(\theta)$$ When $\rho$ increases the least favourable prior sees its support growing, but remaining a finite set of possible values. However the posterior expectation, $\mathbb{E}[\mu|x]$, can take any value on $(-\rho,\rho)$.

The core of the discussion (see comments) may be that, were the Bayes estimator to be constrained to be a point in the support of $\pi(\cdot)$, its properties would be quite different.

Similarly, when considering admissible estimators, Bayes estimators associated with a proper prior on a compact set are usually admissible, although they have a restricted support.

In both cases, the frequentist notion (minimaxity or admissibility) is defined over the possible range of parameters rather that at the "true" value of the parameter (which brings an answer to Question 4.) Furthermore, as pointed out in the above example, when the Bayes estimator is defined by a formal expression such as the posterior mean $$\hat{\theta}^\pi(x)=\int_\Theta \theta\pi(\theta|x)\text{d}\theta$$ for the quadratic (or $L_2$) loss, this estimator may take values outside the support of $\pi$ in cases this support is not convex.

As an aside, when reading

for the true θ to have generated the data (i.e. "exist"), θ must be a possible variate under π, e.g. have non-zero probability, non-zero density

I consider it a misrepresentation of the meaning of a prior. The prior distribution is not supposed to stand for an actual physical (or real) mechanism that saw a parameter value $\theta_0$ generated from $\pi$ followed by an observation $x$ generated from $f(x|\theta_0)$. The prior is a reference measure on the parameter space that incorporates prior information and subjective beliefs about the parameter and that is by no means unique. A Bayesian analysis is always relative to the prior chosen to conduct this Bayesian analysis. Hence, there is not an absolute necessity for the true parameter to belong to the support of $\pi$. Obviously, when this support is a compact connected set, ${\mathscr A}$, any value of the parameter outside the set ${\mathscr A}$ cannot be consistently estimated by the posterior mean $\hat{\theta}^\pi$ but this does not even prevent the estimator to be admissible.