# Exploratory analysis of spatio-temporal forecast errors

The data: I have worked recently on analysing the stochastic properties of a spatio-temporal field of wind power production forecast errors. Formally, it can be said to be a process $$\left (\epsilon^p_{t+h|t} \right )_{t=1\dots,T;\; h=1,\dots,H,\;p=p_1,\dots,p_n}$$ indexed twice in time (with $t$ and $h$) and once in space ($p$) with $H$ being the number of look ahead times (equals something around $24$, regularly sampled) , $T$ being the number of "forecast times" (i.e. times at which the forecast is issued, around 30000 in my case, regularly sampled), and $n$ being a number of spatial positions (not gridded, around 300 in my case). Since this is a weather related process, I also have plenty of weather forecast, analysis, meteorological measurments that can be used.

Question: Can you describe me the exploratory analysis that you would perform on this type of data to understand the nature of the interdependence structure (that might not be linear) of the process in order to propose a fine modelling of it.

• this is a very interesting question. Is it possible to play at least with a subset of anonymized data? And how were the forecasts generated, what kind of model was used? – mpiktas Feb 4 '11 at 8:49
• @mpiktas thanks, you can concider it has been generated with a suitable AR modeling (one for each wind farm), it won't change the problem very much. Sorry, there are too much confidenciality issues with these data, can't provide you anything, even anonymized... – robin girard Feb 4 '11 at 9:57

It seems to me that you have enough data to model the dependence on space-time and meteorological influences of both the bias of forecast errors (i.e. tendency to systematically over-/underestimate [first moment]) and their variance [second moment].

For exploration of the bias, I'd just do a lot of scatterplots, heatmaps or hexbin plots. For exploration of the variability, I'd just square the original errors and then again do a lot of scatterplots, heatmaps or hexbin plots. This is of course not entirely unproblematic if you have lots of bias, but it may still help to see patterns of covariate-influenced heteroskedasticity.

Colleagues of mine did a nice techreport that details a very flexible method for fitting these kind of models (also allows for modelling of higher moments, if necessary) that also has a good R-implementation gamboostLSS based on mboost: Mayr, Andreas; Fenske, Nora; Hofner, Benjamin; Kneib, Thomas and Schmid, Matthias (2010): GAMLSS for high-dimensional data – a flexible approach based on boosting.. Assuming you have access to machines with a lot of RAM (your datasets seems to be BIG), you can estimate all kinds of semiparametric effects (like smooth surface estimators for spatial effects or the joint effect of $t$ and $h$, tensor product splines for tempo-spatial effects or smooth interactions of meteorological effects etc..) for the different moments and perform term selection at the same time in order to get a parsimonious and interpretable model. The hope would be that the terms in this model are sufficient to account for the spatio-temporal autocorrelation structure of the forecast errors, but you should probably check the residuals of these models for autocorrelation (i.e. look at some variograms and ACFs).

• +1 Thanks Fabians, You are totally right, the problem is not that I don't have enough data. Note that my question is especially about the interdependence structure. Scatterplots, heatmaps and hexbin plot are good tools if they are used for the good purpose. I think general additive model can also be very powerfull there is a wonderfull paper of Brillinger providing good hints on how to use GAM. – robin girard Feb 4 '11 at 16:45

We (A colleague and I) finally wrote a paper on that one. To summarized things we proposed two solution to quantify and give a statistical summary of the (spatio-temporal) propagation of errors along Denmark and along look ahead times.

• In the first one we compute the correlation between all pairs of wind farms and for all pairs of look ahead times (this is a function of 4 variables). When a pair is fixed, we showed that the correlation function has a local maxima along look ahead times, we said this is propagation! The temporal scale associated to a given pair of wind farms is given by the temporal lag for wich this local maximum is obtained. Plotting, for all pairs of wind farm the local maxima of correlation, the temporal lag that allows to obtain that and the spatial vector that joins the wind farms gives the right side of Figure 1.

This can be used to compute a global propagation vector i.e. some sort of spatial average of the propagation speeds between pairs. Part of this is shown in left side of Figure 1, and guess what propagation of errors is West East in Denamrk (ok that was not a big surprise :)). We also analysed this conditionally to different meteorological situations in order to show the relationship between propagation and wind (speed,direction).

• The second one is orthogonal (in a sence :)) to the first one. For each time $t$ we fitted a spatio-temporal (along space and look ahead times) plannar wave model with constant propagation speed. This gives one propagation speed per forecast time $t$ (and $R^2$ measuring the quality of the fit obtained by the planar wave model). Then you can compute statistics on those speeds, eventually limiting to the cases when the planar wave fit is good. Results are seen in Figure 2.

In the second case, we observed that the temporal average propagation speed has a similar magnitute as that obtained with the spatial average in the first case. If you want to look at this work more seriously, the paper is here.

• +1 Thanks for sharing. (Sorry I missed the question when it originally appeared.) Did you consider plotting cross-variograms by look-ahead time? The most effective ones would not be the traditional smoothed directional variogram clouds; instead, use two-dimensional plots of the variogram cloud densities. You can then construct cross-variograms of those to explore the temporal relationships. Your propagation results ought to pop out of such an analysis automatically. – whuber Mar 7 '12 at 15:31
• @whuber Thanks for the comment, I hardly believe you have missed more than 2 or 3 question on this site :) . Your idea with the variogramm seems connected (I not very much use to using variogram, I often believe that everything that can be formulated with variogram has a practical equivalent with covariances...), I'll think about it. – robin girard Mar 17 '12 at 12:05
• You are correct that in many applications the covariances are equivalent to variograms. However, the variogram cloud provides both a visual and conceptual supplement that working purely with covariance functions does not seem to offer--it's a little like looking at scatterplots instead of just correlation matrices: you can sometimes see patterns that the numbers do not clearly reveal. – whuber Mar 17 '12 at 14:29