2
$\begingroup$

I need a prior with 2 parameters location (μ) and scale (σ) (a transformation of mean and standard deviation).

My distribution is over discrete values, equally spaced with distance $c$ between consecutive values. Only values between a lower limit $a$ and and upper limit $b$ are considered. For $c\to0$, the distribution becomes continous and when $a\to-\infty$ $\lor$ $b\to\infty$ a truncation point (one or both) is lifted.

A distribution that kind of meet my criteria is a discrete and truncated normal. $P(X = x\vert \mu,\sigma,a,b,c) = \frac{e^{-\frac{(x-\mu)^2}{2\sigma^2}}}{\sum_{y \in \Omega} e^{-\frac{(y-\mu)^2}{2\sigma^2}}}$

Actually, I would prefer fatter tails. I want an uninformative/flat prior (so I can conform to the principle of indifference).

Can I generalise the (discrete) uniform distribution and change the parameters (giving a close form but allowing for unevaluated sums and integrals or other special functions)?

My problem with $\mathrm{Unif} \lbrace a,a+c,a+2c,\dots, b-c,b \rbrace$ is that it doesn't contain μ and σ so I can't calculate my posteriors for the scale and location parameters (I would prefer to calculate directly the posteriors for mean and standard deviation i.e the transformations of my parameters) and calculating a predictive posterior (for the individual scores) with a prior that has different parameters is undesirable.

The values of $a$, $b$ and $c$ will be known every time and are not parameters of interest (they are constants just with different values for each experiment but known beforehand they would be selected/chosen/set by the very researchers I just want the general answer before choosing my constants).

Such a prior would be chosen for example for the individual scores of students in a class (the testmakers decide the minimum grade $a$, the maximum grade $b$ and the granularity $c$) or some other test where the one administering and overseeing sets minimum, maximum and granularity. The individual scores are also believed to be positively correlated (but it is unknown how exactly). For example self-selection, ease of test, good teachers, good time period (e.g covid/no-covid).

I would want a uniform with changed parameters but I am also open to suggestions for alternative priors or methods for conducting my Bayesian analysis.

The best I have got to trying to solve the system of equations for the mean and the variance of the (discrete) uniform is as a PMF

$\dfrac{1}{c\sqrt{12{\sigma}^2+1}-c}$ but I can't get rid of the grandularity ot get the mean I so much want there.

Maybe the problem is that I am trying with an inherently non-parametric distribution.

Edit:

I need to clarify (confess) that I only took 1 cource of Bayesian Statistics (and even that I only passed by the skin of my teeth probably without even having a more than rudimentary grasp of the content out of pity since I was from the School of Economics and I was not studying statistics per se).

I have the belief (maybe false) that it doesn't make sense to choose a uniform prior when you actually know the values of $a,a+c,a+2c,\dots, b-c,b$ because you are the one designing the test. if you know (for certain) a, b and c (you choose them) and "believe" the prior to be $\mathrm{Unif} \lbrace a,a+c,a+2c,\dots, b-c,b \rbrace$ there is no uncertanty left as you know everything.

So when I am not entirely sure that all the items are equiprobable and I don't know the mean nor the variance can I use the uniform as a prior (I only like the uniform because it is what conforms in my view with the principle of indiference)? What exactly is an uninformative prior? How is an uninformative prior different from a uniform prior?

What is the closest thing to an uninformative prior (predictive) supported over the known terms of an arithmetic sequence and with location and scale parametres? Is $\mathrm{Unif} \lbrace 1,2,3,\dots,24,25 \rbrace$ (α=1, b=25, c=1 are known by designing the test) a proper prior, where are μ and σ? Is there any way on making the discrete and truncated normal more uniform maybe try to change the parametres of a student t distribution? I would like to minimise the variance of the y (probability or density) values throught the support (the uniform has equal y for everything so 0 variance there) while keeping the support ($a,a+c,a+2c,\dots, b-c,b$) and the parametres (μ and σ) the same.

Improve this question
$\endgroup$
7
  • $\begingroup$ What do you want your prior over, a prior $p(x)$, as suggested by the fact that you propose a uniform prior, or a prior $p(\mu, \sigma)$? And more generally, what are you trying to do? Do you e.g. ant to infer student ability as well as test difficulty from test results? $\endgroup$
    – all feedback welcome
    Commented Jul 2, 2022 at 20:33
  • $\begingroup$ @allfeedbackwelcome I didn't understand the question What do you want your prior over. Could you please help me with capturing the meaning of the question? I want a prior with 5 parameters (μ,σ,a,b,c) 3 of which (a,b,c) are not actually parametres but constants choosen by the test makers. I am trying to get a posterior on location and scale parametres (or better yet posteriors for the mean and the variance) and calculate a predictive posterior (of possible unobserved individual scores). E.g predicting the individual scores of the students that didn't sit the test. $\endgroup$
    – George Ntoulos
    Commented Jul 2, 2022 at 21:12
  • $\begingroup$ @allfeedbackwelcome In the end I would want to infer student ability but also infer educational effectiveness and apply my findings to future classes (changing my educational policies and even evaluating my teachers, maybe this could lead to a disciplinary sanction against the teacher if the mean is low or the variance is high something is probably wrong with the teacher) $\endgroup$
    – George Ntoulos
    Commented Jul 2, 2022 at 21:16
  • $\begingroup$ @allfeedbackwelcome I most certainly don't want for the probability mass to be proportional to the Gaussian density or that For c→0, the distribution becomes a truncated normal distribution and if additionally a→−∞ and b→∞ it becomes a usual normal distribution. I don't want a connection with Normal (it was continous not truncated normar) Otherwise I already had my Prior the problem is that it is not uniformative-flat (that is why the conditions that you thought I was looking for were not actually what I was looking for). I am looking for a Uniform with changed parametres or for other advice. $\endgroup$
    – George Ntoulos
    Commented Jul 2, 2022 at 21:31
  • $\begingroup$ If you want a posterior over parameters, you also need a prior over them. The prior implies a predictive distribution over scores as well, let me write that down. I write $z:=(\mu,\sigma)$ for the parameters. You have a model $p(x|z)$ of scores given the parameters. (Would you be fine with $z$ being the same for all students?) You need a prior $p(z)$. You can calculate a predictive distribution from the prior by $p(x)=\int p(z)p(x|z)$. Let $D=(x_1,...x_n)$ be the observed scores. You can calculate a parameter posterior $p(z|D)$ and according predictive distribution $p(x|D)=\int p(z|D)p(x|z)$. $\endgroup$
    – all feedback welcome
    Commented Jul 3, 2022 at 17:15

1 Answer 1

Reset to default
1
$\begingroup$

I suspect you have slightly misunderstood how prior distributions work. I'll work through a simplified example.

Let us assume you want to have a uniform (prior) distribution on a parameter $\mu$; the prior distribution looks like:

$$p(\mu; a, b) = {1 \over b-a}$$

This doesn't "contain" $\mu$ because it returns the same value for every input value of $\mu$; that's what "uniform" means. Now, let's show how to calculate a posterior distribution. The likelihood function is Gaussian, and we assume $\sigma$ known:

$$\mathcal{L}(\mu; x, \sigma) = \exp\left\{-{(x-\mu)^2\over 2\sigma^2}\right\}$$

leading to a posterior that looks like:

$$p(\mu; x, \sigma, a, b) \propto {\exp\left\{-{(x-\mu)^2\over 2\sigma^2}\right\} \over b-a}1(a \leq \mu \leq b)$$

where $1(a)$ is the indicator function that takes on the value $1$ when $a$ is true, $0$ otherwise. (You can think of it as a mechanism for reminding ourselves to honor the constraints on $\mu$ imposed by the prior.)

This of course simplifies to:

$$p(\mu; x, \sigma, a, b) \propto \exp\left\{-{(x-\mu)^2\over 2\sigma^2}\right\}1(a \leq \mu \leq b)$$

a truncated Gaussian distribution.

It should be clear how to extend this to the specifics of your problem; you'll just have a discrete uniform instead of the continuous one above, and you'll actually need two of them - one for $\mu$ and one for $\sigma$, I suspect, with different upper and lower limits and different grid step sizes.

Improve this answer
$\endgroup$
23
  • $\begingroup$ I would prefer to work with my prior predictive distribution (the prior for the individual scores not the parametres). What would be my discrete uniform prior predictive? By contrast what would be my normal prior predictive? In fact the upper and lower limits are only the upper and lower limits for individual scores (for the mean they should be honored two and the variance has unknown limits and well the individual scores would be priorly believed to be uniformly distributed there would be no belief on the mean or the variance). $\endgroup$
    – George Ntoulos
    Commented Jul 2, 2022 at 22:04
  • 1
    $\begingroup$ A prior predictive distribution is not a distribution on the parameters. To do Bayesian analysis, you need a prior distribution on the parameters themselves. $\endgroup$
    – jbowman
    Commented Jul 2, 2022 at 22:28
  • $\begingroup$ then probably I don't want to do Bayesian Analysis because I want to focus on the Prior Predictive and Posterior Predictive. I was mistaken I thought that because Priors and Posterios are involved the Bayes Theorem is somehow involved in the calculation and this makes it Bayesian Analysis. In the end is $P(X = x\vert \mu,\sigma,a,b,c) = \frac{e^{-\frac{(x-\mu)^2}{2\sigma^2}}}{\sum_{y \in \Omega} e^{-\frac{(y-\mu)^2}{2\sigma^2}}}$ a proper Prior Predictive and how can I change it into a Uniform with Different Parametres (a,b,c,μ,σ)? $\endgroup$
    – George Ntoulos
    Commented Jul 2, 2022 at 22:53
  • $\begingroup$ What do you think a prior predictive distribution is? How does that differ from just the distribution of the data? $\endgroup$
    – jbowman
    Commented Jul 3, 2022 at 1:42
  • $\begingroup$ A prior predictive distribution is how we believe X (in this case individual scores) are distributed before we look at the data (we have choosen mimimum, maximum grades and the granularity maybe we believe they are normally distributed or maybe believe something else). The distribution of the data is our histogram the graphical represantation of our observations. Bayes theorem provides a way to renew-reinform out beliefs with the data. $\endgroup$
    – George Ntoulos
    Commented Jul 3, 2022 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.