I'm looking for a model between stockprices of energy and the weather. I have the price of the MWatt bought between the countries of Europe, and a lot of values on the weather (Grib files). Each hours on a period of 5 years (2011-2015).

I'm trying to forecast the mean price of the Mwatt.

My data on the weather are very dense, more than 10000 values/hour and so with a high correlation. It's a problem of short,big data.

I've tried the Lasso, Ridge and SVR methods with the mean price of the MWatt as outcome and my weather's data as income. I took 70% as training data and 30% as test. If my test's data are non-forecasting (somwhere inside my training data) I have a good prediction( R² = 0.89). But i want to do forecasting on my data.

So if the test data are chronologically after my trainging data it doesn't predict anything (R²=0.05). I think it's normal because it's a time serie. And ther is a lot of autocorrelation.

I thought that i had to use time serie model like ARIMA. I calculated the order of the method and I tested it. But it doesn't work (the serie is stationary). I mean that the forecasting has a r² of 0.05.My prediction on the test data is not at all on my test data. I tried the ARIMAX method with my weather as regressor. Put it doesn't add any information.

Can i do something more ?