Forecasting and making sense of data This might seem very elementary or even silly, but all comments are much appreciated. As science major I do not have deep technical knowledge in statistics, so any guidance is much appreciated.
Say we have this hypothetical problem:
Say I would like to come up with a model to predict returns of some financial instrument $I$ for a next day. I have the return values $R_n$ of the $I$ for the past $n$ days.
Additionally for each day $d$ I have some independent data $P_d$ (say $m$ numbers) that could be used to predict return value $R_d$ of today, but I have no idea what this data means and where it came from.
Are there any standard ways of finding how data $P_d$ relates to $R_d$, for example some kind of software package that loops through many possible models and check how well they fit?
Or what would be a way to tackle this type of problems, because it seems to me there could be infinite number of ways to use $P_d$ to predict $R_d$?
 A: Actually, I was thinking of posting a similar question, though coming from a different angle.
Firstly, your data $R_d$ and $P_d$ form a time series with potential interactions and you might end up using a vector autoregreesion methon (as in https://www.otexts.org/fpp/9/2)
In that case, you have to analyse all the auto-correlations and so on. 
Additionally, with a high dimensional $P_d$ you will get time series in a very high dimensional space. 
So far I have no experience in the time-series regime, but this seems not very feasible.
Hence, one could also use a regression task that is fed with your data.
You still might so enrich you data $P_d$ with additional features.
An advantage might be to use a nonlinear model, since they might find some structure in the data you don't see directly.
For example a neural network can be used as is pointed out in here or in here
You can also use a random forest as is pointed out here
Note that here an optimization is performed to obtain the best parameters for moving averages that contain the data from previous events/time-steps
