# Spatio-temporal convolutional networks on irregular grids

I'm contemplating a project where I try to take a time series of maps of polygons (which have values) and predict the next map of polygon values.

If it were a regular grid, it'd be a straightforward to apply a TCN with a 3D convolution, and causal padding on the time dimension. But the grid is irregular.

It seems like the simplest thing to do would be to define a grid, and assign the centroids of the polygons to the nearest gridpoint, padding the rest with zeros. Depending on resolution of the grid I'd have a lot of sparsity, but I think that Keras can handle that pretty well, right?

Anyway, I'd appreciate suggestions on ways of dealing with this irregular topology. I see papers on generalizing CNNs to graphs, but those don't seem super well-developed. Are any of those methods well-suited to this purpose, are there other good methods for predicting spatial time series on irregular grids, or is my sparse grid idea a reasonable hack?

You could try the "reverse" of what you suggested: For each gridpoint in your grid, assign it the value of the polygon it is contained in. This has the advantage of reducing the amount of sparsity. Then to get the predicted value of a polygon in the next time step, just take the mean/median across all gridpoints contained in that polygon.

Another thing which may or may not help is to pass in a 4-channel "mask" which contains a 4-coloring of your polygons -- this would help the network out with learning the border between polygons when the input values in two adjacent polygons are the same / very similar.

About the mask. I suppose it's importance may depend on whether the polygon map is always the same, or whether it can change between different inputs. In the latter case, if the input consists of 3 rectangular polygons with the same value, then it may look like this to the network

2 2 2 2
2 2 2 2
2 2 2 2
2 2 2 2


Which is why I suggest feeding in an additional mask separating the polygons as shown below:

0 0 0 0
0 0 0 0
1 1 2 2
1 1 2 2


A one hot encoding of the above map, which need only ever have 4 distinct values due to the four color theorem, would indicate to the network the boundaries between adjacent polygons.

• Thanks for the suggestion. I had thought about this, but rejected it because I'd be over-weighting larger polygons. But I suppose that I could get around this issue by making some sort of layer that averages over the polygon, as you suggest. Could you elaborate on the "mask" idea? To be clear, most of my covariates are measured at the polygon level, rather than over continuous space. – generic_user Feb 1 at 20:54
• @generic_user I explained the rationale for a mask a bit more. – shimao Feb 1 at 21:16
• Clever idea with the mask and with the 4-color theorem (first I'd heard of that). But as I think about it, wouldn't it run into problems if the spatial receptive field is big enough to capture multiple non-adjacent polygons that are given the same color? Or perhaps in that case the number of colors is yet another hyperparameter? – generic_user Feb 1 at 21:32