How does the fixed point interation in invertible resnets work?

Question

I feel like I am missing some easy point about this invertible resnet paper which is making it hard for me to grasp how the fixed point iteration works.

stated simply, the residual connection in a resnet architecture is of the form $y = x + f(x)$. Therefore, one could run things in reverse to get $x = y - f(x)$. The main point of the paper is that if $f(.)$ is a contraction mapping, meaning that the Lipschitz constant is less than 1, $$ || f(x) - f(x') || < k|| x - x' || \;\; \forall x \in X $$

then we can do a fixed point iteration where $x_0 = y$ and $x_{t+1} = y - f(x_t)$ which will converge to $x = y - f(x)$.

Question

I understand that a contraction mapping must converge to a fixed point via the Banach Fixed Point Theorem. What I do not understand is why the fixed point in this case will converge to $f(x)$. It states in Section 2 above Lemma 2 that

Note, that the starting value for the fixed-point iteration can be any vector, because the fixed-point is unique. However, using the output $y = x+g(x)$ as the initialization $x_0 := y$ is a good starting point since $y$ was obtained from $x$ only via a bounded perturbation of the identity.

So why is it the case that the contraction point is exactly at $x$ and not some other arbitrary point in the space?

Answer 1

The point of the inversion is to find the $x$ that generated a given $y$ of the form $y = x + g(x) = F(x)$, right?

(I have kept the notation of the paper, specifically, that $F(x)$ denotes the whole residual block, while $g(x)$ is used to denote the sequential parameterized layers.)

Now, take a look at the fixed point iteration step: $$ x_{t+1} = y - g(x_{t}) $$ and assume convergence at the $n$-th iteration, i.e., $x_{n} = x_{n+1} = x^*$

If you rewrite and rearrange the last iteration step

$$ x^* = y - g(x^*) $$ $$ y = x^* + g(x^*) $$ $$ y = F(x^*) $$

you can express $y$ precisely as the output of the residual block when the input is $x^*$. Therefore, $x = x^*$, so the fixed point iteration converges (always - under Theorem 1) to the (unique) fixed point that is the inversion of $y = F(x)$.

Hope it helps.