When teaching regression, I used to do an exercise where students would try to guess where the line of best fit is on a scatterplot, and get the sums of squares. They'd move the line around, see how the slope and intercept changed, and see how the sum of squared residuals changed.
It shows that the line of best fit really does minimize the sums of squares, and it's fun, because you have a little competition to see who can get the lowest sum of squares.
(It also teaches a little about how iterative approaches work.)
This relied on a horrible, horrible feature of Excel - that you could click on a graph and drag a point, and it would update the data. (Not that we need a reason to dislike Excel for data analysis).
I've a vague memory of seeing a java (possibly) app, many years ago that did this, but I can't find it now. Is there something else out there?