6
$\begingroup$

I'm a physics undergrad who started becoming curious about this question after exam season. After any exam, we're typically given the following parameters: Min, max, mean, median, std. deviation, and the total number of students who took the test.

Clearly this isn't enough information to determine the distribution of scores precisely, but I wonder if there's a way to construct the 'simplest' distribution that is consistent with the parameters. For example, start with a normal distribution with the given mean & std. deviation, and then try to introduce skew to make it fit? I could write a computer program run this optimization, but I wonder if there's a more analytical way to do this?

Any ideas on how to try to find the optimum distribution in light of a normal 'prior'?

$\endgroup$
2

2 Answers 2

12
$\begingroup$

Obviously as a starting point to this kind of question, we must recognise that there are may possible probability distributions on the relevant support that have the same (finite) set of moments. Here you have, say, 101 possible outcomes for the grade but only two fixed moments, so you have two equations in one-hundred unknowns (once we subtract a degree-of-freedom for the norming constraint of probability). If you add in the additional constraints from the order statistics then you have five equations in one-hundred unknowns This means that any method to construct a "simplest" distribution is typically going to involve some method of quantifying the "simplicity" of any possible distribution and then an optimisation of that quantity over the possible distributions.

One well-known approach to this problem is the technique of finding the maximum entropy distribution given a set of moment constraints (and the support for the random variable). In this method, the distribution is chosen by maximising the entropy under the given constraints. Since entropy represents the amount of "information" or "uncertainty" in a distribution, you might reasonably regard this as a proxy for "simplicity".

Suppose that the possible marks on this test are $0,...,M$ and you have $n$ students, so that the sample distribution can be represented from a count vector $\mathbf{n} = (n_0,...,n_M)$ with $\sum n_i = n$. The entropy of the distribution is:

$$H(\mathbf{p}) \equiv \sum_{i=0}^M \frac{n_i}{n} \cdot \log \bigg( \frac{n_i}{n} \bigg).$$

Your sample constraints are knowledge of the statistics $x_{(1)}$, $x_{((n+1)/2)}$, $x_{(n)}$, $\bar{x}_n$ and $s_n^2$. All of these can be written as functions of the count vector $\mathbf{n}$, which means that you have five constraining equations on the count vector.$\dagger$ To find the maximum entropy distribution, your optimisation problem would be:

$$\text{Maximise } H(\mathbf{p}) \text{ subject to constraints }$$

Unfortunately, since four of your five constraints are nonlinear, this is an extremely complicated nonlinear discrete optimisation problem. Unless the sample size is small, it may be extremely difficult to compute any count vector that exactly satisfies the contraints, let alone find if there is more than one point in the optimisation set. Consequently, you will need to give consideration to finding an algorithm to maximise the entropy subject to those constraints and you will need to decide whether you want to relax things (perhaps by looking at a continuous approximation that allows noninteger counts) to simplify the optimisation problem. In any case, setting aside the computational difficulties of the optimisation problem, maximum-entropy is one reasonable way to optimise for a "representative" distribution that obeys the relevant constraints.


$\dagger$ Here I have assumed an odd-number of data points for simplicity in computing the median. You will need to make appropriate changes to the median formula if there are an even number of data points.

$\endgroup$
5
  • 1
    $\begingroup$ "setting aside the computational difficulties of the optimisation problem, this will give you a reasonable method" seems to be saying "it is difficult but, ignoring that issue, it is easy". If the possible marks are integers from $0$ to $M$, the exact standard deviation requirement may make finding any solution less than simple, let alone an optimal solution $\endgroup$
    – Henry
    Commented Jun 19, 2023 at 10:53
  • 2
    $\begingroup$ @Henry The theoretical understanding needed to come up with the entropy approach is highly non-trivial. You can think of the problem as a two step process, this provides an excellent answer for the first part and tells you what to look for to solve the second part. $\endgroup$
    – quarague
    Commented Jun 19, 2023 at 10:56
  • $\begingroup$ @Henry: Yes, that is true. I have revised the answer to stress that the discrete nonlinear optimisation would be infeasible beyond small samples and so a continuous approximation may be required. In any case, my main point in this answer is that the criterion of maximum-entropy might be fit for the purpose that the OP is seeking. $\endgroup$
    – Ben
    Commented Jun 19, 2023 at 12:31
  • $\begingroup$ I am somewhat bemused as maximum entropy distributions are associated with theoretical moments, not empirical one. For instance, what would the theoretical equivalent to the maximal observation be? $\endgroup$
    – Xi'an
    Commented Jun 19, 2023 at 17:09
  • $\begingroup$ The problem of finding the maximum entropy distribution is not so difficult if we restrict to 4 conditions. E.g. the maximum entropy distribution with given range (min/max), mean and standard deviation is a truncated normal distribution. More difficult will be to incorporate the median as well, maybe some sort of mixture between a truncated Gaussian and Laplace distribution? $\endgroup$ Commented Jun 19, 2023 at 19:44
2
$\begingroup$

As a simple-minded guy, I'd ask the physics dept if their standard post-exam reporting could be upgraded to include a histogram and plot of the ordered scores, and/or an Excel file of the individual test scores for curious DIY-ers.

This would allow everyone to see the ACTUAL distribution of exam scores, including any weirdities, rather than some estimated "most likely" theoretical distribution.

And you could figure out how you did on a ranked or percentile basis, relative to the other test-takers. That could possibly be useful in calibrating your "intensity of effort" for the rest of the duration of the class.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.