Here, the data $x$ are randomly generated, and $t$ are generated by running $x$ through a function $\sin(2\pi x)$, then Gaussian noise is added.
Bishop's text then tries to fit those data using a polynomial of 9th degree, i.e., $y(x,w) = w_o + w_1 x + \ldots + w_Mx^M$ (Shown in figure)
Then they showed that this results in large mean squared error for test data.
And the reason being that the weights associated with each of the parameters is extremely large, something like $w_i = O(1000000)$
But is this a fluke?
Why does the polynomial's weight need to be so large?
A polynomial of 9th order needs to have 8 bumps.
That's all it needs. So Bishop's curve could simply look like the black curve which I drawn.
No more huge weights, problem solved.
Can someone comment on why must the weights associated with the polynomial be so large when I fit a high order polynomial, instead of having smaller weight like the curve I drew?
Is this simply a simulation error in Bishop's text, or does solving the optimization problem actually give you weight that large (which I do not believe)?
Added: Exact weights as shown in textbook
Experiment: Poly-fitting in R using same data