Suppose I have a set of $k$ matrices.

$$ \mathbb D = A_1,A_2,...,A_k $$

Each column of $A$ is categorical vector.

$$ A = v_1,v_2,...,v_n $$

I want to find the mapping $$ f: A \rightarrow Q $$ Where $Q$ is a $(n,n)$ matrix such that

$$ \frac{1}{k} \sum_k \text{MSE}\left(Q_{k,real}(ij) - Q_{k,predict}(ij)\right) $$ is minimized.

MSE is chosen because $Q(ij) \in 0,1$ representing some continuous event occuring.

Thus far several distance metrics $d(A(i),A(j))$ have been directly used to form the elements of $Q(i,j)$. They are not parameterized in anyway by the bins of the frequency vector or bins of the cross-tabs of the $v$s. Chi-squared gave bad results. I settled on symmetric uncertainty(information theory). Forming $Q_{predict}$ it easy to show that the error can be greatly minimized by Z-scoring it assuming a bivariate Gaussian distribution, however the results are less than spectacular. (This feels wrong and there is still a lot of error.)

I'm looking for a well studied supervised machine-learning/fitted-model for the entire process, accounting for the graphical structure and cross-tabulation data. So far I've heard suggestions from computer science people on Belief Nets, recurrent neural networks, structured SVM and multiple correspondence analysis. Multiple correspondence analysis is the only method though expect nominal data, but it doesn't fit anything nor model the joint PMF. Ideally I would like the model to be interpretable and not a black-box(like SVM). I'm much better at linear algebra and programming than I am at statistics. I've thought about building a nested model and solving it with something like stochastic gradient descent. I'm willing to go experimental on any suggestions. I have an abundance of CPU cycles and training data but no good model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.