I am investigating the relationship between temperature fields obtained from numerical weather models and electricity demand. I am applying a PCA-based approach, i.e. I study the linear relationship between main temperature patterns/modes and main demand patterns/modes. Given that I am working on summer yearly demand, I have time-series with few samples (<20) and for this reason I've decided to apply the following bootstrapping procedure:
- I create a temperature and elec. demand datasets with the usual sampling with replacement
- I create my linear model between the two fields
- I create a temperature dataset with the not-selected samples and I project them on the PCA-space I've just computed
- I calculate out-of-sample output
I do this for about 5000 times and in the end I obtain a matrix with only the out-of-sample outputs. I calculate the mean on all the out-of-sample predicted demands for each year and I use it to calculate RMSE error. I think this approach could be considered a .632 bootstrap procedure.
I'd like to compute the significance of the obtained results. I was thinking about the possibility to shuffle at each bootstap iteration the temperature dataset in order to see whether I obtain similar results breaking the direct temporal link between demand and temperature.
Given that I don't have a robust statistical background I'd like your opinion about any method to obtain the statistical significance of my bootstrap procedure.