# Matrix decomposition with constraints and weighted least squares

We have a matrix, $$\mathbf{X}$$, of probability distributions between 6 different results, so each row $$\mathbf{x}_i$$ sums to 1. We want to perform dimension reduction so that each row is a linear interpolation of two components, $$\mathbf{f}$$ and $$\mathbf{g}$$, which are also probability distributions. In other words, each row $$\mathbf{x}_i$$ has a latent variable $$c_i$$ so that $$\mathbf{x}_i \sim c_i \mathbf{f} + (1 - c_i) \mathbf{g}$$. Furthermore, we have the following constraints:

• Certain probabilities in the components are 0:
• $$\mathbf{f}$$ is of the form $$(f_0, 0, f_2, f_3, f_4, f_5)$$.
• $$\mathbf{g}$$ is of the form $$(0, g_1, g_2, g_3, g_4, 0)$$.
• As probability distributions, $$\mathbf{f}$$ and $$\mathbf{g}$$ also sum to 1.

In addition, some results are more important than others to estimate accurately, so we would like to weight the least squares error: Let the estimate of row $$x_i$$ be $$\hat{x}_i = c_i \mathbf{f} + (1 - c_i) \mathbf{g}$$. We would like to minimize the weighted sum of squared residual, $$\sum_{j=0}^{5} w_j (x_j - \hat{x}_j)^2.$$ I'm currently working in Python but am open to other tools to solve this problem. Thank you.

Context: For the game of Blaseball, for each player, we have counts of 6 events when the ball is in-play: flyout, ground_out, single, double, triple, and home_run. We hypothesize a generative model where first either a fly or a grounder occurs, and then the corresponding probability distribution, $$\mathbf{f}$$ or $$\mathbf{g}$$, is used to generate the observed event. Each player has a latent variable that describes their probability of selecting either $$\mathbf{f}$$ or $$\mathbf{g}$$. The constraint of 0 in certain indices of $$\mathbf{f}$$ and $$\mathbf{g}$$ is the assumption that certain generator-observation pairs are impossible, i.e., a fly cannot result in a ground_out, a grounder cannot result in a flyout or home_run.

• A natural tool is a probabilistic programming language for statistical inference, such as Stan. Choose some suitable probability distribution over probability vectors $D_f$ and $D_g$; an example is the dirichlet-distribution. To model the constraint, set the distribution's parameters to values that assign minuscule probability to the values you want to be 0.
– Sycorax
Commented May 12 at 4:42
• would you mind adding to your description why you want to make these constraints, and what your overall purpose is? Commented May 12 at 8:47
• I have added some context to explain the constraints, thanks Commented May 13 at 18:43

i would think this could be done relatively easily using an unconstrained nonlinear least squares optimiser such as levenberg marquardt

you can turn your probability constraints on f_j, into unconstrained variables,z_j, using a 'multinomial transformation'

model $$f_j =\exp(z_j)/(1+\sum _k exp(z_k))$$

where the sum is from 0 to 4, and f_5 is calculated as 1 - that sum.

and similarly for the gs and cs

scipy provides a non linear constrained optimisation function https://docs.scipy.org/doc/scipy-1.13.0/reference/generated/scipy.optimize.least_squares.html

so you just constrain f_0 to f_4 to lie between 0 and 1 and f_5 is computed as 1- sum.