# Is a Restricted Boltzmann Machine appropriate for predicting a vector?

I have a very large set of animal migration data, consisting of many series of vectors - each series is basically a path of a single animal. The dataset I'm using consists of 244 of these series.

I want to train a generative algorithm so that when it is given a collection of these series and a map of environmental variables, such as a map of ocean surface temperature, it can develop a model of how an animal reacts to these variables; when I feed it new variables, it would generate a new set of vectors.

My question is which machine learning algorithm would be optimal for this kind of prediction - I was looking into multinomial logistic regression, but that is for categorical data. I want the algorithm to develop a model that thinks "at a point with an ocean surface temperature of 20˚C, an animal will travel along a vector with a magnitude of 35km and an angle of 34˚".

Can a Restricted Boltzmann Machine be used to develop this kind of model? I would train the RBM with a set of vectors, each with environmental variables coupled to the head and tail of the vector - would the RBM be able to create a new set of vectors if I then fed in different environmental variables?

• This model would not take the location (e.g. proximity to the land) into account. Or is it also going to be an environmental variable? Commented Oct 16, 2015 at 13:36
• I think that the proximity to land would be an environmental variable - would this prevent any points from showing up on land? Commented Oct 16, 2015 at 20:05

Woaw, this is a really interesting problem. Based on what you write below it does not seem like you want a generative algorithm, but instead a discriminative algorithm (see this post). A generative algorithm would think "given a magnitude of 35km and an angle of 34˚, the temperature was likely 20˚C".

I want the algorithm to develop a model that thinks "at a point with an ocean surface temperature of 20˚C, an animal will travel along a vector with a magnitude of 35km and an angle of 34˚".

If you think it would make sense to just model the relationship between temperature and direction (perhaps with magnitude) then you could use all sorts of regression methods. However, this doesn't take into account the current location, the path that the animal has moved for the last days etc. In general, I think you should really consider how you pose the problem. Instead of having two different representations (a list of paths and a map of surface temperature), I would try to have a single representation. Similarly, be clear about the wanted output from the model. Training deep neural networks for regression is difficult - especially if you want to output two continuous variables (angle + magnitude). Instead you could make the map into a grid and then model the choice of going in any of the 8 directions on the grid.

This could be modeled it using a recurrent neural network (RNN) or a variant of this called a long short-term memory (LSTM) network. These networks are fed lists of variable size to predict an outcome (see 'many-to-one classification' here). As input you would have a list of observations, where each observation contains e.g. the following info: (time, map_location, ocean temperature). To generate a path given a temperature map and a starting position you would in an iterative manner input the current path and temperature to the RNN, output the next move and then do it again. Something very similar is already done to generate text using RNN and LSTM.

• Thanks! But say I didn't want to use a neural network any more, but I just wanted to use some sort of regression algorithm: would this make it easier? I just really want to know which algorithm (no particular preference) is the best for this problem. Commented Oct 16, 2015 at 20:06
• As I try to explain in my post, that completely depends on how exactly you define the problem :) Do you only want to examine which direction an animal goes given a certain temperature? Or do you also want to include things like the path that they've traveled so far etc.?
– pir
Commented Oct 16, 2015 at 20:17
• But yes, it would probably be easier to start off with a "simple" technique such as logistic regression or random forest. However, to do that you still need to split your data and represent it in the right way. All in all, it is really important that you think about what problem you want to solve and then go from there. Included in 'defining the problem' is the aspect of how much information you want to include in your data representation and how.
– pir
Commented Oct 16, 2015 at 20:19
• Basically I want to take into account the animal's path as a whole, and see how that differs from the predicted path. The ultimate goal is to see how the entire migration pattern of a population changes, so I would eventually compare all of the paths as a whole to the predicted paths. I was thinking of using the random forest algorithm, but that is for classification - that wouldn't work for predicting the two numerical properties of the travel vector. Commented Oct 16, 2015 at 20:32
• Random forest can also be used for regression. You could model the two numbers separately although I'm not sure how much information you actually get from predicting those two properties.
– pir
Commented Oct 23, 2015 at 22:09