In [chapter 8][1] section 8.7.1 it tries to explain batch normalization. In the second paragraph of that section it tells us to consider the simple example:

$$ \hat{y} =  x w_1 ... w_i ... w_l $$

and then claims:

> The output $\hat y$ is a linear function of the input x, but a nonlinear
> function of the weights $w_i$.

which I believe is incorrect, however, I wanted to make sure I was not wrong myself. I will argue here why I think its wrong.

Recall the definition of a linear function to be $ f(x+y) = f(x) + f(y) $. Now lets consider the function they wrote and see if it obeys that property. First it clearly obeys it with respect to x:

$$ \hat{y}(x) + \hat{y}(y) = x w_1 ... w_i ... w_l + y w_1 ... w_i ... w_l $$

then by factoring out $w_1 ... w_i ... w_l$ we get:

$$ \hat{y}(x) + \hat{y}(y) = ( x + y ) w_1 ... w_i ... w_l = \hat{y}(x + y) $$

One can do nearly an identical proof but with respect to $w_i$:

$$\hat{y}(w_i) + \hat{y}(w'_i) = x w_1 ... w_i ... w_l + x w_1 ... w'_i ... w_l $$

but instead by factoring everything first from the left and then from the right. Similarly to why $abc+ab'c = a(bc + b'c) = a(b + b')c$ is true. Do that and we get:

$$\hat{y}(w_i) + \hat{y}(w'_i) = (x w_1 ... )(w_i ... w_l + w'_i ... w_l) =  (x w_1 ... )(w_i + w'_i)( ... w_l) = \hat{y}(w_i + w'_i)$$

which yields the desired result (that $\hat y$ linear wrt to $w_i$). 

From the above argument I can't see why they'd say its non-linear. Maybe I have a misunderstanding what they are trying to say? Or is there a small mistake on the draft of the book? If its not to much to ask, can a potential answer try to address **why** is my proof is **wrong**?




  [1]: http://www.deeplearningbook.org/contents/optimization.html