I'm looking for a model between stockprices of energy and the weather. I have the price of the MWatt bought between the countries of Europe, and a lot of values on the weather (Grib files). Each hours on a period of 5 years (2011-2015).

Price/day

This is per day for one year. I have this per hours on 5 years.

Example of weather

3Dscatterplot, in kelvin, for an hour

I'm trying to forecast the mean price per hour of the Mwatt.

My data on the weather are very dense, more than 10000 values/hour and so with a high correlation. It's a problem of short,big data.

I've tried the Lasso, Ridge and SVR methods with the mean price of the MWatt as outcome and my weather's data as income. I took 70% as training data and 30% as test. If my test's data are non-forecasting (somwhere inside my training data) I have a good prediction( R² = 0.89). But i want to do forecasting on my data.

So if the test data are chronologically after my training data it doesn't predict anything (R²=0.05). I think it's normal because it's a time serie. And there is a lot of autocorrelation.

I thought that i had to use time serie model like ARIMA. I calculated the order of the method (the serie is stationary) and I tested it. But it doesn't work . I mean that the forecasting has a r² of 0.05.My prediction on the test data is not at all on my test data. I tried the ARIMAX method with my weather as regressor. Put it doesn't add any information.

##ACF/PCF, Test/train data

Can I do something more ?

Maybe I've done overfitting.