Binary version of Probabilistic Matrix Factorization in pymc?

Question

I'm a newby in statistics, this is my first post, sorry for any possible mistake.

There is a good Bayesian Probabilistic Matrix Factorization model introduced in: Bayesian Probabilistic Matrix Factorization using Markov Chain Monte Carlo

I was trying to implement a Binary version of it, i.e., all the Entries in the rating matrix is 0 or 1, you may think of it as indicating whether each rating is missing or not. So I guess the latent vectors U and V should also be changed to Binary.

Now, since I'm not good at statistics, my guess is that I should make the following change on the generative model of 1:

Change the Multivariate Gaussian Priors on U,V into a set of Bernoulli priors, e.g., for a latent vector $U_i$, its elements are deemed to be generated from

$$ U_{i,k} \sim \mathrm{Bernoulli}(\pi_k) \\ \pi_k \sim \mathrm{Beta}(\alpha,\beta) $$

$k=1...D$, $D$ is the dimension of latent factors, $\alpha,\beta$ are certain hyperparameters.

Is this the right model? If so, is it possible to implement its MCMC parameter estimation using pymc? Just show me a simple outline of the pymc code will be perfect.

---

This is a huge problem for me. Thanks in advance!

Answer 1

You don't necessarily need to change $U$ and $V$ to be binary as well; in fact, doing so will probably be somewhat harmful. Remember that $\mathbb{E} R_{ij} = \sum_k U_{ik} V_{jk}$, so that if $U$ and $V$ have binary entries, for $\mathbb{E} R_{ij}$ to be 0 or 1, at most one corresponding pair of entries can be 1.

One thing you could do is just to use the BPMF model as-is, probably centering your data (subtracting the mean) beforehand. I would expect this to work reasonably well.

A better option might be to keep the same distributions on $U$ and $V$, but change the mean of $R$ to be "squashed" through, say, a logistic function $\mathbb{E} R_{ij} = 1/(1 + \exp(- U_i^T V_j)$ (and still modeling R as normal); this is what was done in the original (non-MCMC) PMF paper. This makes the model predictions not actually binary, but at least they're in the right range. I'm not sure how easily pymc allows you to model this (I've never used it).

A more "pure" option, to keep the data types of the models correct, would be to model $R$ as Bernoulli with parameter based on $U_i^T V_j$; probably squash it as above, or at least make sure that in your code it's truncated into [0, 1].

I don't know how the different options will affect the efficacy of the simple Metropolis-Hastings algorithm that the current version of pymc uses. I believe it would be less of an issue with the more powerful algorithms in the alpha version (pymc3), but I have no idea how stable or usable that code is yet.

Answer 2

According to this post, it seems I have to give up pymc --- The ratings are 'determinitic' given U and V, while pymc only allows observations to be 'stochastic', maybe that's why I can't find any pymc implementation of PMF out there.

This maybe useful for someone trying to use pymc in MF like me.