Let's suppose the following toy example: we are given the task of estimating how many years a person has yet to leave.
For this problem we have tabular data such as age, height, ethnicity, etc; and we also have various pictures from a given person. Furthermore, let's suppose I trained all tabular information into a random forest and got a decent result, I also trained a neural network on the dataset but got worst results than with the random forest.
Considering that we have, say, from 2 to 10 pictures of the person, I am looking for a way to boost the prediction from the random forest considering the pictures for a given person, e.g., training a convolutional NN.
Assuming that the pictures bear information about our final task, I now have two models: one for the pictures and another one for the tabular data. Since a given person can have more more than one picture,
- How would I go about training a convolutional NN that considers various samples from a person and returns an averaged prediction over the pictures.
- How can I combine my random forest with my newly created ConvNN?