Why is polynomial regression used to demonstrate overfitting and underfitting? WhenI try to research overfitting and underfitting, the most common algorithm and explanation I see revolves around polynomial regression. Why is this so? Is it just because it can be easily visualised like the graphs here:
https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfitting_overfitting.html
Why is polynomial regression deemed suitable to give an insight into these overfitting/underfitting concepts? Are there any other similar algorithms that could be used?
 A: A more colloquial answer to this would be that the polynomial regression, the higher the order, has more flexibility to learn but also the tendency to memorize every data point in the training set, but with unneccessary and unpredictable wriggly waves, that makes forecasting nearly impossible as algorithms can get stuck in one area and extrapolation is quite problematic. Thus polynomial regression memorizes data wich is equal to overfitting, and does not learn to deal with new data, it only catches a certain situation. And as Stephan highlighted, the visualization helps a lot here.
Update just to make it clear:
Overfit just means a model that learns its data very well, which can be very well shown with a polynomial. Actually we wouldn't need a train/test split to see it, because our mind should say that we cant explain something so perfectly. Look how the polynom tries to catch every data point in both pics. Just to make it more clear due to the comments below my post, I added another picture:
ML Overfitting:

Picture by Jannis Seemann Udemy

*

*The overfitting polynom is the red line.

*Grey crosses are train data

*Blue crosses are test data

*Green line is a simpler model, (e.g. a power function) that doesnt catch the data exactly, but learns the general trend or a picture of a model situation.

You can see that the red polynom learnt the data very well, but behaves bad on the test data. The simpler model, not catches the whole train data but behaves well overall. If you are aware of this behavior you wont need a visual train/test split to be sure if your model overfits, if you choose to a high polynom.
Polynom behavior in general
This picture shows a general comparison of a overfitted model with a high order polynom and a polynom of a much lower order. You can already see, that the model with the higher order overfits, although you havent seen the test data.
To feel the power of a simple overfit, you can do pictures like that yourself:
Use a simple scatterplot. Got to Microsoft excel, click on fitting the scatter plot with a linear trend. Than change to polynomial und increase polynomials to the highest order. The polynomial gives you the freedom to fit the data near perfectly, but that squiggly line does not normally fit on the test data, its totally useless (blue one). The red line although a much lower fit, may be extrapolated better and shows "maybe" the truth, that there is not much to expect, Your model will be bad, which is part of antoher discussion, but not overfitted.

Image by researchgate https://www.researchgate.net/profile/Scott-James-5/publication/255205930/figure/fig1/AS:670698729123859@1536918446824/The-green-squares-represent-the-original-data-set-are-fit-using-two-conceptual-models-a.png
A: This is a less statsy answer, and more of a practical/human-oriented answer. While all the other answers are correct in their own manner. I think it is more to do with your audience. Let me explain:
If you're learning about overfitting then you're going to be pretty new to ML/Stats/Data Science. Therefore, not everyone taking said class, reading said blog post or watching said video is going to understand more complex models like GBMs, SVMs, GMMs and so on...
However, if you are learning ML you should sure as hell will understand the concept of linear regression (or line of best fit) and one would hope you'd understand polynomials (otherwise maybe you should be taking high school maths classes not trying to learn ML just yet).
Finally, you can create and recover the generative model precisely. This, in a learning phase, helps people fully understand what is going on and allow them to easy replicate results and play with the models themselves.

NB: Maybe the real question is "Why aren't more ML concepts introduced as polynomial regression examples, since they are so intuitive, expressive and accessible by everyone with a high school mathematics education?".
The honest answer to this is probably: "Because GBMs, SVMs, GMMs etc are freaking cool!".
A: Easy visualization is a huge point in favor of using polynomial regression for illustration. (Note that both "illustration" and "demonstration", etymologically, have to do with showing pictures!) It also helps that the degree of the polynomial controls the amount of overfitting, and that polynomial regression allows looking at bona fide nonlinearity in the relationship (although splines are a better way of dealing with this).
An alternative that is also commonly employed is to simply add spurious predictors. Your true data generating process might be $y=b_1x_1+b_2x_2+\epsilon$, so a model $y=\beta_1x_1$ is underfitted, $y=\beta_1x_1+\beta_2x_2$ is fitted correctly, and $y=\beta_1x_1+\beta_2x_2+\beta_3x_3$ is overfitted - but it's hard to visualize the increased variability of fits from the overfitted model.
