I'm fitting something similar to a naive Bayes model to a data set where each data point has six features, $A_1$, $B_1$, $C_1$, $A_2$, $B_2$ and $C_2$. $A_1$ and $A_2$ can both take values in {$a_{1}$, $a_{2}$, $a_{3}$, ...}, $B_1$ and $B_2$ can take values {$b_1$, $b_2$, ...} and similar for C.

The important part here is that the {$A_1$, $B_1$, $C_1$} labels could be swapped around with {$A_2$, $B_2$, $C_2$} and still describe the same observable. Whereas only swapping $A_1$ and $A_2$ without swapping the others would not. So

$$ P(A_1=a_i, B_1=b_j, C_1=c_k, A_2=a_l, B_2=a_m, C_2=a_n) = P(A_1=a_l, B_1=b_m, C_1=c_n, A_2=a_i, B_2=a_j, C_2=a_k) $$ and $$ P(A_1=a_i, B_1=b_j, C_1=c_k, A_2=a_l, B_2=a_m, C_2=a_n) \neq P(A_1=a_i, B_1=b_m, C_1=c_k, A_2=a_l, B_2=a_j, C_2=a_n) $$

(Technically $A_1$ can't take the same value as $A_2$, but I don't think that that will end up being too important).

The easy way to account for this in the model is to add a copy of the data set where the set labels have been interchanged and then just carry on as this symmetry didn't exist. In the naive bayes case you would have six categorical nodes. Alternatively you could just make a joint categorical variable that describes the set {$A_1$, $A_2$, etc.}, which would also make sure that $A_1 \neq A_2$, but the amount of variables would blow up due to the second requirement above.

So my question is whether there's a better way of handling these types of selective permutation invariance other than extending the dataset with the possible permutations?

EDIT: Clarified some things.

  • $\begingroup$ Could you be more specific about what you mean by "model" these variables? Your language seems to construe that either as replicating some data, making some "fit" to some unspecified model, or perhaps something else. $\endgroup$ – whuber Feb 13 '18 at 13:58
  • $\begingroup$ @whuber Thanks for pointing out that I was unclear. Have hopefully cleared it up a bit. $\endgroup$ – user2653663 Feb 13 '18 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.