We have a matrix, $\mathbf{X}$, of probability distributions between 6 different results, so each row $\mathbf{x}_i$ sums to 1. We want to perform dimension reduction so that each row is a linear interpolation of two components, $\mathbf{f}$ and $\mathbf{g}$, which are also probability distributions. In other words, each row $\mathbf{x}_i$ has a latent variable $c_i$ so that $\mathbf{x}_i \sim c_i \mathbf{f} + (1 - c_i) \mathbf{g}$. Furthermore, we have the following constraints:

  • Certain probabilities in the components are 0:
    • $\mathbf{f}$ is of the form $(f_0, 0, f_2, f_3, f_4, f_5)$.
    • $\mathbf{g}$ is of the form $(0, g_1, g_2, g_3, g_4, 0)$.
  • As probability distributions, $\mathbf{f}$ and $\mathbf{g}$ also sum to 1.

In addition, some results are more important than others to estimate accurately, so we would like to weight the least squares error: Let the estimate of row $x_i$ be $\hat{x}_i = c_i \mathbf{f} + (1 - c_i) \mathbf{g}$. We would like to minimize the weighted sum of squared residual, $$\sum_{j=0}^{5} w_j (x_j - \hat{x}_j)^2.$$ I'm currently working in Python but am open to other tools to solve this problem. Thank you.

Context: For the game of Blaseball, for each player, we have counts of 6 events when the ball is in-play: flyout, ground_out, single, double, triple, and home_run. We hypothesize a generative model where first either a fly or a grounder occurs, and then the corresponding probability distribution, $\mathbf{f}$ or $\mathbf{g}$, is used to generate the observed event. Each player has a latent variable that describes their probability of selecting either $\mathbf{f}$ or $\mathbf{g}$. The constraint of 0 in certain indices of $\mathbf{f}$ and $\mathbf{g}$ is the assumption that certain generator-observation pairs are impossible, i.e., a fly cannot result in a ground_out, a grounder cannot result in a flyout or home_run.

  • 1
    $\begingroup$ A natural tool is a probabilistic programming language for statistical inference, such as Stan. Choose some suitable probability distribution over probability vectors $D_f$ and $D_g$; an example is the dirichlet-distribution. To model the constraint, set the distribution's parameters to values that assign minuscule probability to the values you want to be 0. $\endgroup$
    – Sycorax
    Commented May 12 at 4:42
  • $\begingroup$ would you mind adding to your description why you want to make these constraints, and what your overall purpose is? $\endgroup$
    – seanv507
    Commented May 12 at 8:47
  • 1
    $\begingroup$ I have added some context to explain the constraints, thanks $\endgroup$
    – jgf1123
    Commented May 13 at 18:43

1 Answer 1


i would think this could be done relatively easily using an unconstrained nonlinear least squares optimiser such as levenberg marquardt

you can turn your probability constraints on f_j, into unconstrained variables,z_j, using a 'multinomial transformation'

model $f_j =\exp(z_j)/(1+\sum _k exp(z_k))$

where the sum is from 0 to 4, and f_5 is calculated as 1 - that sum.

and similarly for the gs and cs

scipy provides a non linear constrained optimisation function https://docs.scipy.org/doc/scipy-1.13.0/reference/generated/scipy.optimize.least_squares.html

so you just constrain f_0 to f_4 to lie between 0 and 1 and f_5 is computed as 1- sum.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.