# Neural networks assume continuity, what does this mean?

I encountered the following paragraph by Pedro Domingos (mentioned in Gary F. Marcus paper):

ANNs assume continuity, graphical models assume conditional independence, and instance-based learning assumes similarity; and correspondingly, neural nets make it easy to incorporate types of continuity like translation invariance, graphical models [make it easy to incorporate] conditional independences, and [instance-based models make it easy to incorporate] knowledge of what makes things similar (in the kernel or distance measure, which will vary with the domain).

I understand conceptually what is translation invariance, but I don't understand what is meant by continuity and why translation invariance is a kind of continuity.

If you implement a translation function: $$x' = \text{translate}(x, \Delta)$$ which takes image $$x$$ and translation amount $$\Delta$$, using bilinear interpolation, then it will be continuous wrt both $$x$$ and $$\Delta$$. Let $$f(\cdot)$$ be a continuous neural network. Then $$g(x,\Delta) = f(\text{translate}(x, \Delta))$$ is continuous wrt both $$x$$ and $$\Delta$$ since continuity is closed under composition.