Probability Mass Function making the Truncated Normal Discrete

Question

I want to discretize the Truncated Normal to describe the spread in the Grades of students in a class sitting the same exam.

Let an arbitrary number of students sit the same exam. The grades are usually said to be normally distrubutted (they couldn't possibly be distributed with the truncated normal and much less by a simply normal because the grade are bound from both ends and are discrete).

The minimum grade is a parametre a (even negative grades are allowed)

The maximum grade is a parametre b

The minimum difference possible between two grades is a parametre c

x (measured in X axis) is the grade

y (measured in Y axis) is the probability of a student achieving X grade or the frequency that students in the class achieve grade X.

μ is a transformation of the mean, it is the mode if μ lies between a and b and is also the mean if μ lies halfway between a and b

σ is a transformation of the SD, it is the SD when $a\to-\infty\cap b\to\infty \cap c=0$

When $c=0$ we have a Truncated Normal

When $a\to-\infty\cap b\to\infty \cap c=0$ we have a Normal.

I want my distribution (f(x,μ,σ,a,b,c)) to meet some criteria

1 Being supported on the terms of an arithmetic progression. Both the initial and final terms and the common difference are parametres

2 Being unimodal in all cases but 1. Being bimodal in case the mode is equidistant from 2 values in the support.

3 Having probabilites strictly decreasing as X distances itself from the mode. The decrease in probability should be equal for x equally distant from the mode. The change in the difference should first decrease and then increase after reaching a quasi-inflection point. The change in the difference should also be equal for x equally distant from the mean. I.e if the mode coincides with the midrange the distribution is symmetrical while if the mode is closer to one bound the distribution is symmetrical for all values closer than the nearest to the mode bound (short of like the Truncated Normal if you fold it at the mode all the values that are closer that the nearest bound coincide 1 to 1 and only what is further away does not coincide because there is nothing left for them to correspond to)

Hopefully the first 3 criteria are all compatible with each other.

4 If the above three criteria allow for it the distribution should belong to the Natural Exponential Family (better) or the Exponential Family (Plan B)

5 The distribution should be the Maximum entropy distribution that the above 4 criteria allow for (ignore criterion 4 if the first 3 make it impossible for the distribution to belong to the Exponential Family)

Such that the probability mass assigned to any point is proportional to the kernel of the normal distribution (leaving the mean and the standard error a parametre) calculated at that point.

E.g when a=0, b=25 and c=1

$f(0,\mu,\sigma, 0, 25, 1)\propto\mathrm{e}^{-\frac{{\mu}^2}{2{\sigma}^2}}$

$f(1,\mu,\sigma, 0, 25, 1)\propto\mathrm{e}^{-\frac{\left({\mu}-1\right)^2}{2{\sigma}^2}}$

$f(2,\mu,\sigma, 0, 25, 1)\propto\mathrm{e}^{-\frac{\left({\mu}-2\right)^2}{2{\sigma}^2}}$

$f(3,\mu,\sigma, 0, 25, 1)\propto\mathrm{e}^{-\frac{\left({\mu}-3\right)^2}{2{\sigma}^2}}$

.

.

.

$f(25,\mu,\sigma, 0, 25, 1)\propto\mathrm{e}^{-\frac{\left({\mu}-25\right)^2}{2{\sigma}^2}}$

of course

$\displaystyle \sum_{x=0}^{25} f(x,\mu,\sigma, 0, 25, 1) = 1$

or more generally

$\displaystyle \sum_{x=a}^b f(x,\mu,\sigma, a, b, c)=1$

What is the closed form solution (probably in terms of Theta functions)

Edit.

I am hoping the probability mass function I will get will be a conjugate prior (in the same probability family) that a posterior predictive distribution would be if we started with a discrete uniform prior (with a common difference c between each adjacent value in the support) where the random variable would be the individual students performance and each grade would be assigned the same frequency (if there are say 25 values in the support, 25 different grades possible, it would assign a probability of 0.04 for each grade) and the observations would be the individual performance of every student in the class (in the end we will have the performance of the class not simply their mean but an histogram). The posterior would be a distribution assigning a probability that a student randomly chosen scored X given that the class scored as the histogram shows.

I don't understand the difference between

$$P(X = x\vert \mu,\sigma,a,b,c) = \frac{e^{-\frac{(x-\mu)^2}{2\sigma^2}}}{\sum_{y \in \Omega} e^{-\frac{(y-\mu)^2}{2\sigma^2}}}$$

and

$$p_X(x) = \frac{\Phi \Big( \frac{x - \mu + 1/2}{\sigma} \Big) - \Phi \Big( \frac{x - \mu - 1/2}{\sigma} \Big)}{\Phi \Big( \frac{m - \mu + 1/2}{\sigma} \Big) - \Phi \Big( \frac{1/2 - \mu}{\sigma} \Big)} \quad \quad \quad \text{for } x=1,...,m,$$

Whuber said that they are not closed form solutions, because they involve an unevaluated sum. There is no closed form solution that doesn't involve Theta functions. I don't understand this either.

Answer 1

When you use the values of the normal density then you get automatically the first four conditions satisfied

$$P(X = x\vert \mu,\sigma,a,b,c) = \frac{e^{-\frac{(x-\mu)^2}{2\sigma^2}}}{\sum_{y \in \Omega} e^{-\frac{(y-\mu)^2}{2\sigma^2}}}$$

where $\Omega = \lbrace a,a+c,a+2c,\dots b-c,c \rbrace$ all the values in the support.

---

Maximum entropy

The fifth condition, maximum entropy, is also fulfilled since the maximum entropy distribution with constraints $\sum_{\forall x} x p(x) = \mu$ and $\sum_{\forall x} x^2 p(x) = \text{var}$ must be of the form $$P(X=x) = ce^{\lambda_1 x + \lambda_2 x^2}$$ which is like the distribution above (e.g. see that with $\lambda_1 = \mu/\sigma^2$ and $\lambda_2 = -1/(2\sigma^2)$ you get the above).

---

We can also prove it explicitly by considering the Kullback-Leiber divergence or Gibbs inequality with another distribution $f(x)$, and our distribution $g(x)$

$$\begin{array}{} - \sum_{\forall x} f(x)\log f(x) &\leq& - \sum_{\forall x} f(x)\log g(x) \\ &\leq& - \sum_{\forall x} f(x)\log \left(ce^{\lambda_1 x + \lambda_2 x^2} \right)\\ &\leq & - \sum_{\forall x} f(x)\left(\log c + \lambda_1 x + \lambda_2 x^2 \right)\\ &\leq & - \sum_{\forall x} g(x)\left(\log c + \lambda_1 x + \lambda_2 x^2 \right) \\ &\leq& - \sum_{\forall x} g(x)\log g(x) \end{array}$$

This second last step where we switch from $f(x)$ to $g(x)$ is because of the constraints $$\begin{array}{} \sum_{\forall x} f(x) &=& \sum_{\forall x} g(x) &=& 1 \\ \sum_{\forall x} xf(x) &=& \sum_{\forall x} xg(x) &=& \mu \\ \sum_{\forall x} x^2f(x) &=& \sum_{\forall x} x^2g(x) &=& \text{var} \end{array}$$ with these constraints we can rewrite $$\sum_{\forall x} f(x)\left(\log c + \lambda_1 x + \lambda_2 x^2 \right) = \sum_{\forall x} g(x)\left(\log c + \lambda_1 x + \lambda_2 x^2 \right)$$

So we must have that the entropy of $f(x)$ must be smaller than the entropy of $g(x)$.

Answer 2

The other answer here uses the normal density values at the exact points. Another similar method would be to take the normal probabilities across intervals centred on those points. In the latter case, taking the support to be $X = 1,...,m$ you get:

$$p_X(x) = \frac{\Phi \Big( \frac{x - \mu + 1/2}{\sigma} \Big) - \Phi \Big( \frac{x - \mu - 1/2}{\sigma} \Big)}{\Phi \Big( \frac{m - \mu + 1/2}{\sigma} \Big) - \Phi \Big( \frac{1/2 - \mu}{\sigma} \Big)} \quad \quad \quad \text{for } x=1,...,m,$$

where $\Phi$ is the CDF of the standard normal distribution. You can then adjust to an arbitrary arithmetic progression by taking the appropriate linear transformation. This has the first three properties you stipulated in your question.

Answer 3

The most natural way would be as follows:

Let $X$ have support $0,1,...T$, where $T \in \{2,3,...,\}$ (i.e T can be taken as the limit to $\infty$, since the infinite sum can be bounded by the integral over the kernel of a gaussian pdf, but it will be intractable to work with in practice).

$$f_X(X=x;\mu, \sigma) = \frac{exp(-\frac{(\mu -x)^2}{2\sigma^2})}{\sum_{y=0}^Texp(-\frac{(\mu -y)^2}{2\sigma^2})}$$

This is a special case of the softmax function ($\frac{exp(z_i)}{\sum_{i=1}^Nexp(z_i)})$, which is commonly used to map the reals to the interval $(0,1)$ with the property that the elements sum to 1. There are many choice models which take this form, such as the multinomial logit, with $T < \infty$.

I agree with @Xi'an, however, that defining a pmf this way is unlikely to be justified. It is hard to imagine a principled construction of what this would represent and as Xi'an stated, the properties of a gaussian that make it nice to work with are not inherited by this new model.