0
$\begingroup$

I am working on an image prediction problem, where we use a U-Net to predict a real-valued image. I've found that conventional metrics like MSE, r^2, MAE, etc just don't really cut it. What are some of the spatial verification techniques that are used today? I use python, by the way. There is more information about spatial verification here, and there are methods too.

I wonder if there are methods I am not yet aware of. I would also prefer to use a python package, but can code up the metric myself if need be.

Note: I tagged image-segmentation because it is related to my prediction problem, but here we are working on regression while image-segmentation is a classification problem.

Edit: I've found a great package for spatial analysis in R; SpatialVx by NCAR: https://projects.ral.ucar.edu/icp/SpatialVx/

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ can you describe why conventional metrics aren't working for you? $\endgroup$
    – shimao
    Commented Sep 30, 2021 at 2:14
  • $\begingroup$ Sure, please see the first paragraph in this paper arxiv.org/pdf/2106.09757.pdf Essentially, for the purposes of meteorology, it's not so much about hits and misses on individual pixels, but how closely the general pattern matches the observed pattern. I also found resources on R, which I will post here in an answer. $\endgroup$
    – McM
    Commented Sep 30, 2021 at 17:56
  • $\begingroup$ Check out the earth mover's distance (Wasserstein metric). It considers how far (spatially) one has to move "mass" (your real valued signal) in order to have the prediction match the truth. $\endgroup$
    – bogovicj
    Commented Sep 30, 2021 at 19:47

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.