The square root of $r^2$ is in effect the correlation between observed values and those predicted by the polynomial. (That's why it is denoted $r^2$.)
Whether the polynomial is a better fit than a straight line is a more challenging question. A polynomial with quadratic and possibly other terms can wiggle much more than a straight line but whether it is a better idea overall is harder to say.
Positively, a polynomial can capture important features of a relationship that a straight line just can't match, such as a turning point. Negatively, and this is the advantage reversed, a polynomial can wiggle in ways that make no substantive sense and/or would be treacherous if taken literally or extrapolated incautiously.
What you should always do includes
Comparing $r^2$ from a polynomial fit with the square of plain Pearson's $r$ to see how much improvement is implied by the polynomial.
Plotting the polynomial fit together with the original data to gauge the improvement.
Thinking about the substantive interpretation of the polynomial fit, especially whether explicit or implicit turning points, inflections and axis crossings make sense.
(I find it hard to imagine that a Bayesian would say anything different in spirit.)