0
$\begingroup$

In a particular Bayesian problem, I have encountered a choice of parameters that leads to a uniform posterior distribution. Given prior

\begin{equation} p(\boldsymbol{\pi}) =Dirichlet(\boldsymbol{\alpha}), \quad \text{with}\ \boldsymbol{\alpha} = [0,\dots,0]. \end{equation}

we consider a multinomial distribution whose support is $\{x_1, x_2, \dots, x_n\}$ and $\text{Pr}(X=x_i)=\pi_i(\sum_i\pi_i=1)$

\begin{equation} \label{eq:likelihood} p(X|\boldsymbol{\pi}) = \mathcal{M}ulti_{k}(n,\boldsymbol{\pi}), \end{equation}

we find the posterior \begin{align*} p(\boldsymbol{\pi}|X) &\propto p(X|\boldsymbol{\pi})p(\boldsymbol{\pi})\\ & \propto \prod_{i}\pi_i\prod_{i}\pi_{i}^{\alpha_i-1}\\ & \propto \prod_i\pi_i^{\alpha_i}. \end{align*}

to be a uniform distribution

\begin{equation} p(\boldsymbol{\pi}|X) =Dirichlet(\boldsymbol{\alpha}), \quad \text{with}\ \boldsymbol{\alpha} = [1,\dots,1]. \end{equation}

I was wondering if it is uniform, why is this called a posterior then? How can we have a posterior distribution that is a uniform distribution?

$\endgroup$
  • 1
    $\begingroup$ All (continuous) posterior distributions are uniform when their variables are suitably expressed. (This is known as the Probability Integral Transform.) $\endgroup$ – whuber Aug 13 at 16:19
  • $\begingroup$ @whuber: However, the reparameterisation would most likely depend on the observable, which would then make it an object with no prior! $\endgroup$ – Xi'an Aug 13 at 16:46
  • 1
    $\begingroup$ @Xi'an Perhaps. But--especially for discrete data distributions with large-probability modes--given a prior, one could anticipate the most likely posterior and before observing the data choose a parameterization for which that posterior has a uniform distribution. Indeed, that very well could be what the OP has inadvertently done ;-). $\endgroup$ – whuber Aug 13 at 17:01
  • 1
    $\begingroup$ Although you might have a likelihood, you do not know the distribution of the observations: that's the thing you're trying to discover! $\endgroup$ – whuber Aug 13 at 18:23
  • 1
    $\begingroup$ You aren't reading my comments, then, because I don't use "buzzwords" and I don't use "uninformative." The restated question in your comment is completely different than the question you actually posted because you failed to mention the "does not depend on data" part. $\endgroup$ – whuber Aug 14 at 14:09
5
$\begingroup$

NOTE: My answer was written based on an earlier formulation of the question, namely on how having a uniform Dirichlet $\mathcal D(1,\ldots,1)$ posterior was at all possible. I proposed this setting as an illustration of a possible case. The question later got edited and, behold!, coincided with this case.

Consider the special following case of

  1. an improper prior on $\mathbf p=(p_1,\ldots,p_k)\in\mathfrak S_k$, the $k$-th dimensional simplex, proportional to$$\pi(\mathbf p)\propto \prod_{i=1}^k p_i^{-1}\tag{1}$$
  2. an observation $\mathbf x=(x_1,\ldots,x_k)$ of $\mathbf X\sim\mathcal M_k(k;\mathbf p)$ equal to $(1,\ldots,1)$
  3. a posterior equal to the Dirichlet $\mathcal D(1,\ldots,1)$ distribution

To comment on this special case,

  1. the posterior obviously depends on the data. Were the data $\mathbf x=(0,\ldots,0,k)$ the posterior would be another Dirichlet distribution. Any other observation than $(1,\ldots,1)$ does not produce a Uniform posterior (with this prior (1)).
  2. A Uniform posterior is a distribution and as such contains information about the probability parameter $\mathbf p$. It is limited because the data is limited but increasing the number of observations will see the posterior concentrate. There is nothing paradoxical with having a uniform prior with a single observation, at least in a sampling distribution over a finite set. Unless the likelihood is constant in the parameter, this is impossible in a continuous setting.
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Yes, you read my mind! I wanted to make it clear that my question was not on give an example (maybe I didn't word it correctly at first). The discussion on the comments are closer to what I'm misunderstanding. $\endgroup$ – Blade Aug 13 at 18:20
  • $\begingroup$ I think second point should be modified as follows: "an observation $\mathbf x=(x_1,\ldots,x_k)$ of $\mathbf X\sim\mathcal M_k(k;\mathbf p)$ with support on $\mathbf d=\left(d_{1}, \ldots, d_{K}\right)$ and $\mathbf n=(1,\ldots,1)$ where $n_k$ the number of $x_{i}$ equal to $d_{k}$". Reading yours gave me the impression that the actual observations are $(1,\ldots,1)$, while it is the number of repetitions of each possible observation $n_k$ that should be $1$. $\endgroup$ – Blade Aug 15 at 17:50

Not the answer you're looking for? Browse other questions tagged or ask your own question.