I'm interested in modeling a generative process that encourages data to be "evenly distributed" over its support, i.e. clumping of data points is penalized.
For example, if I have a mixture distribution and I set all components to have equal weight, it will tend to generate points equally between all the components, but this property isn't enforced. The problem here is that if we evaluate a data set under this likelihood, it gives the same probability to a data set that is evenly distributed as it would to a data set that is clumped under a single component.
To illustrate a "better" model that does penalize clumping, consider the following generative process. Beginning with a mixture distribution with $N$ components, sample a point in the standard way: first sample an index $i$ from the component weights, and then sample a point from that component. Now decrease the weight of the $i$-th component to be a certain percentage of its original weight (call this percentage $p$), and distribute that mass to the other $N-1$ components. Repeat $M$ times.
This process generates points from the mixture distribution in a way that encourages points to be evenly distributed between the components. The parameter $p$ adjusts how strongly this even-distribution of points is enforced--when $p=1$, this is a typical mixture distribution. When $p=0$ and $M = N$, this enforces a strict one-to-one correspondence between data points and mixture components. The problem with this process is that although it's straightforward to sample from, evaluating the data likelihood under this model appears to be intractable, due to the need to consider all possible orders in which the data could have been selected.
Are there any well-known models that penalize clumping in this way?