1
$\begingroup$

I am using machine learning to create a land use regression model. My inputs are geographic coordinates. These I use to extract 80x80 meter satellite images or maps to feed the model.

Lets take the example of a neural network:

Existence: The solution will always exist for any finite input. It may overflow if the input and/or weights are unreasonable large, but the solution exists in principle.

Unique: The solution will be unique by definition. Of course, the training data may have non-unique outputs for their inputs due to measurement noise, but the resulting model will only produce one output given an input.

Continuity: Hadamards last criterion I am uncertain how to adjust to land use regression. While neural networks are function compositions of continuous functions and thus continuous themselves, the input space is not; The border between various object, like houses and streets, form discontinuities.

Sure, if I deform what's inside the image continuously, there is no issue, but how can I allow, lets say, a new street continuously enter the image? If I start scanning over a street, and I do it as a function of time, there will be a time $t_n$ in which the image $I(t_n)$ does not contain the street but after which $I(t_n +\delta)$ will include the street for all $\delta>0$ (as long as $\delta$ is not so large that the street has left on the opposite side of the image). This is a discontinuity that would lead to discontinuous output as well, and thus contradict Hadamards final property.

What is the correct way to think about this if I want it to be well-posed? I mean, I guess I can consider my inputs as a local statistic of sorts while my actual input is the entire globe... But, how would one go about making land-use regression well posed? Or for that sake, any problem where the input is a rectangular subset of Earth as seen from space.

$\endgroup$
9
  • $\begingroup$ "If I start scanning over a street..." I have the idea now that you have a series of nearby images in which some have the street en some have not. But I am not sure what the problem is with this. How does the mechanism of the machine-learning work? What is it doing? And what is the deal with the time, can I imagine that an airplane or satellite is scanning along some path making repeatedly video/picture frames of 80 by 80 meters? So the time is effectively a coordinate in space? Or are you scanning the same place and observe changes in time? $\endgroup$ Commented Sep 25, 2021 at 8:43
  • $\begingroup$ @SextusEmpiricus The point here is not the task or how it's presented; It's Hadamards third criterion for well-posedness; That is, if I continuously move across the globe, the output should change continuously as well. It doesn't really matter much what the task here is; Any sudden discontinuity in the input space, here it is the globe as seen from above, seems to contradict that property. I'll try rephrasing the question. It may be a tad misleading. $\endgroup$
    – Avatrin
    Commented Sep 25, 2021 at 9:45
  • $\begingroup$ Maybe my issue with this question is that I haven't heard of Hadamard's criterions before. I could also not find any articles that use it or at least a search for Hadamard's criterion on JSTOR and Google scholar only gave unrelated results. What I don't see is how you believe that discrete input is problatic. This is ubiquitous in machine learning. E.g. any image recognition algorithm will be trained with images that either do or don't contain a cat in the image. Continuous cat's live only in thoughtproblems in quantum mechanics. $\endgroup$ Commented Sep 25, 2021 at 14:17
  • $\begingroup$ @SextusEmpiricus I never said one that discreteness is the issue. Where did you get that from? I wrote continuity, so clearly I am thinking of the underlying uncountably infinite set. $\endgroup$
    – Avatrin
    Commented Sep 25, 2021 at 14:25
  • $\begingroup$ "That is, if I continuously move across the globe, the output should change continuously as well. " Why should the output change continuosly? Is this some criterion for landuse regression? For an average statistics user this doesn't sound obvious. $\endgroup$ Commented Sep 25, 2021 at 14:27

1 Answer 1

2
+50
$\begingroup$

You could imagine a "sliding view function" $f(I, x) = V$ of some large input image $I$ and position $x$, and returns a cropped view $V$ of the image. This function could be implemented with linear interpolation*, which allows $x$ to be any continuous value -- in particular, this allows you to shift the view by arbitrarily small $\epsilon$. Since $f$ is continuous/differentiable wrt $x$, $\text{NN}(f(I,x))$ is also continuous wrt x.

*In one dimension, linear interpolation says that if pixel $i$ in the output image is centered on a non-integer position $x$ in the input image, then letting $u = \lfloor x \rfloor, v=\lceil x \rceil$, the value of pixel $i$ should be $I(u)(v-x) + I(v)(x-u)$. So each pixel is continuous wrt $x$.

You might think it doesn't make any sense that the entirety of earth fits in some image $I$ / there's no reason to map the earth onto some discretized equirectangular grid. In that case, imagine $g(x)$ to be "the image captured if I put a camera at location $x$". Let's examine some arbitrary fixed pixel in this image: if we had an ideal pinhole camera, and objects in real life had sharp boundaries, then yes, the value of this pixel could be discontinuous wrt $x$. In practice, there is a point spread function that says even an perfectly sharp point source of light gets blurred out by an imaging system. So I claim $g(x)$ will be continuous.

$\endgroup$
2
  • $\begingroup$ Your second paragraph is quite good, but I don't see the accuracy in your first paragraph. How is f continuous wrt x? And, how do you define g? $\endgroup$
    – Avatrin
    Commented Sep 25, 2021 at 16:25
  • $\begingroup$ @Avatrin I elaborated a bit on linear interpolation, so hopefully it should be clear now. $\endgroup$
    – shimao
    Commented Sep 25, 2021 at 20:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.