Fitting a linear relationship, not a standard linear regression model (in Python)

Question

I want to learn about the relationship between 3 categorical variables (a, b, c). I have pairs of such triples that appear together. Ie.:

$$ ((a_1, b_5, c_3), (a_3, b_5, c_7)) $$$$ ((a_2, b_2, c_1), (a_2, b_8, c_3)) $$$$ ((a_4, b_1, c_6), (a_8, b_1, c_2)) $$$$ ... $$

I would like to fit a linear model that assigns a weight to each category like:

$$ x_{i1} \cdot y_{j1} \cdot z_{k1} + \epsilon_{some} ~= x_{i2} \cdot y_{j2} \cdot z_{k2} + \epsilon_{another} $$

where $x_i$ is the weight assigned to $a_i$, $y_i$ is the weight assigned to $b_i$, and $z_i$ is the weight assigned to $c_i$.

And then learn all these weights.

What would be a good way to do this in Python? Maybe statsmodels, or what other options do I have? Maybe sample my way out of it using PyMC?

In particular the triples correspond to clothing brand, type and size. I after fittings the model I want to be able to predict one of the variables (size in particular) given the 5 other. I wanted to start with something very basic that would work on variables that was never seen together, but feel free to come up with suggestions on how to model this in at better way. The size is a mix of many different sizing charts, so for now I will treat them as deprecate categories, later I would like to "translate" them into a latent variable.

Answer 1

I don't think "linear" is a helpful word to describe this problem.

It sounds to me like you're looking for a set of weights, $x_i, y_j, z_k$ such that the product of the weights included in each member of a pair are as close to each other as possible. That is, your datapoints give us relationships like:

$(x_1y_5z_3)-(x_3y_5z_7)=\epsilon_1$

$(x_2y_2z_1)-(x_2y_8z_3)=\epsilon_2$

And we want to minimize $\sum_r |\epsilon_r|$ or $\sum_r \epsilon_r^2$. This objective function is just a complicated sum of products of the variables, i.e. we can think about it as the function $f(\vec{x},\vec{y},\vec{z})$.

I assume we have some other restrictions on the weights, or the solution $x_i=y_j=z_k=0\ \forall i,j,k$ will trivially satisfy that the pairs are equivalent.

I'm not aware of any optimization technique that naturally fits the structure of this problem. We can throw any standard method for optimizing nonlinear functions, like gradient descent or genetic algorithms or tabu search or so on, against it but it's not obvious that any of them will perform particularly well.