Suppose you are fitting a 1-degree polynomial to 2 points- i.e., a line through 2 points. There is one and only one answer. If you fit (least squares) a line through 3,...,n points, there is also only one line, although the line can't go through all the points, it just finds a 'best place' for fitting the line among the points (again, least squares).
Now suppose you want to fit a quadratic (2-degree polynomial) to those same 2 points. Many quadratics will fit those two points exactly- an infinite number, in fact. Many cubics will also, many quartics, etc. So there isn't one solution, there are infinitely many solutions in all those cases. That's why your equation-solving software spits up at that point. n points can be fitted exactly by an n-1 degree polynomial, and in an approximate (least squares again) way by a polynomial of degree 1,...,n-2. But a polynomial of degree n will fit all those n points exactly, but with 'wiggle room left over'- there will be infinitely many degree n polynomials which will fit. To decide on which one of those infinitely many solutions you would want, you'd need to have some additional criteria besides just the points. The higher the polynomial degree, the more criteria you would need. And your typical fitting software would have to work differently. This begins to get a bit complex.
(But for the most part, you don't want a really high-degree polynomial to fit your data anyway- you're hoping you can explain a lot by fairly simple equations)