I'm wondering if there is any methodology or work that illustrates learning on different types of data, but with the same goal/prediction task.

As an example, if I have two different data sources: text and images. Let's also suppose that text and images share some subset of features (e.g. metadata). My task would be to predict if given an example $i$, where $i$ can be an image or text, whether the user will like example $i$.

So as a first thought, I could just train two different models, one for text and one for images, but would there be some better approach where we can use information from both?


Instead of training a model to predict, based on the content of the image or text, that an unknown user would like the content, a content-based recommender filter, consider a collaborative-based recommender.

A collaborative-based recommender does not need to know or care that an item is text, image, a car, a video, a video game, it will predict a users preference for an item based on the users preferences for other items that similar users have a preference for.

For example criticker.com uses collaborative filtering to make recommendations on both movies AND video games without knowing any details of either other than the preferences the user has. Take a look at their Taste compatibility index for an example of how it works.

  • $\begingroup$ a collaborative recommender can still be improved by adding a neural network on top of the results, like with an auto-encoder to find a mapping of the similarity metric to a lower number of dimensions and, presumably, give better results. $\endgroup$ – Brenda.ZMPOV Oct 16 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.