I have many satellite raster images available from different sensors. From these, the coarser ones have a very abundant temporal resolution. The medium resolution rasters tend to have less acquisition dates but still some degree of information is available. The finer resolution ones have a very low temporal resolution, spanning from 2 to 6 observed dates in under two years. I was wondering if anyone knows of any efforts to study this type of multi-scale time series in any way? I would be interested in predicting future values at the finer scales using the information available from the coarser ones. It makes sense to me that the data must be related (yes the images cover the same regions) but I have no idea how to start coupling this information in a predictive model.
2 Answers
Spatial Domain:
It seems more like an image processing problem to me. Clustering methods might help but which metric (distance, variance,discontiguity...) and which algorithm (k-means, mean-shift, EM...) is best fit in your case is determined by your image topology and features you are going to use. You may implement the image binning on medium and fine rasters. Then try different clustering techniques to see which one gives you the overall best segmentation accuracy compared with your original medium/fine rasters. Some pre-processing strategies in order to find the scale space hierarchy might help. There is one hierarchy segmentation algorithm shown in Chapter 3 of this report in which you
(1) Build a scale space;
(2) Find the extrema and saddles at every scale level;
(3) Link each critical point at a certain scale level to its corresponding location at the next scale level, and find the critical paths;
(4) Scale space hierarchy determination based on the iso-intensity surface searching.
For the clustering methods that the random initialization is required, such as k-means, you can use the found hierarchy as the initial clusters and centroid for further clustering. Besides, depending on the characters of your image, you may also want to add more features (such as texture changes, other space information than RGB space, etc) in clustering algorithms.
Temporal Domain
Now you have the images with different time scale but the same resolution (hopefully). If your prediction job is estimate the movement of some of the continent, storms, or precipitation, you may try motion estimation with Kalman filter. The motion for each pixel can be weighted inside the corresponding region(cluster) based on its metric compared with the centroid of the region. You can use neural network for short-term time sequence forecasting (chapter 3 in this thesis). And since Kalman filter is simply a method for implementing Bayes rule, the maximum likelihood can be applied for state estimation. State-estimation procedures can be implemented recursively. The posterior from the previous time step is run through the dynamics model and becomes the new prior for the current time step. Then this prior can be converted into a new posterior by using the current observation. As a result, iterative parameter re-estimation procedures such as EM can be used to learn the parameters in Kalman filter. Chapter 6 of the same thesis, and the study on Kalman smoothing both include more details on the parameters learning with EM.
You should look into the literature for super-resolution. This area typically solves the problem of taking in multiple coarse resolution images to create one high resolution image by borrowing strength across multiple images effectively.
I've listed some relevant literature that should be a good starting point.
My favorite approach here uses nonlocal means. This involves splitting all the images up into patches of $5x5$ or $7x7$ pixels, creating better estimates of pixels in the finer resolution image using a weighted combination of pixels in the coarser images.
References
Elad, Michael, and Arie Feuer. "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images." Image Processing, IEEE Transactions on 6.12 (1997): 1646-1658.
Park, Sung Cheol, Min Kyu Park, and Moon Gi Kang. "Super-resolution image reconstruction: a technical overview." Signal Processing Magazine, IEEE 20.3 (2003): 21-36.
Protter, Matan, et al. "Generalizing the nonlocal-means to super-resolution reconstruction." Image Processing, IEEE Transactions on 18.1 (2009): 36-51.
