76
$\begingroup$

The ancient greeks famously sought to construct geometrical relationships using only a ruler and a compass. Given a set of points in a two dimensional plane, is it possible to find the OLS line using only such instruments?

This question has absolutely no practical application that I can think of.

$\endgroup$
11
  • 1
    $\begingroup$ I was wondering if a physical device based on strings could be a practical implementation but Hooke's law states that the force scales linearly with the distance rather than as its square... $\endgroup$ – Xi'an Feb 1 at 5:56
  • 2
    $\begingroup$ @Xi'an Is that a problem, though? Since the force scales linearly, the potential energy scales with the square of distance. A physical contraption will move towards a configuration that minimises its kinetic energy - which is to say, the sum of squares of spring displacements, weighted by spring stiffness - so it should be quite possible to build a spring-based line-of-best-fit machine. $\endgroup$ – Geoffrey Brent Feb 1 at 9:58
  • 1
    $\begingroup$ @Xi'an it is not, you would end up implementing PCA, not linear regression (unless the springs are limited to "vertical" movement only) $\endgroup$ – Firebug Feb 1 at 15:18
  • 1
    $\begingroup$ I was indeed thinking of imposing vertical forces! $\endgroup$ – Xi'an Feb 1 at 15:44
  • 1
    $\begingroup$ @Xi'an Rubber bands don't quite follow Hooke's law, because when compressed they don't exert much restoring force. If you wanted to use those, you'd need something like opposing pairs of bands so that they're always in tension, as well as something to impose the vertical-only movement restriction. But it should be doable. $\endgroup$ – Geoffrey Brent Feb 2 at 1:25
102
$\begingroup$

Loosely speaking, it's apparently possible to compute any quantity which can be expressed "using only the integers 0 and 1 and the operations for addition, subtraction, multiplication, division, and square roots" with only a compass and ruler -- the wikipedia article on constructible numbers has more details. Since the slope of the OLS line definitely has such a closed form, we could deduce it's possible to construct the line.

As someone who isn't an expert in compass and ruler constructions, I found this a bit unbelievable, so I gave it a try myself: the green line is the OLS fit for the three blue points, not fitting an intercept for simplicity.

You can play around with it here for yourself and drag around the blue points a bit.

enter image description here

Here's roughly how the construction went: it turns out you can multiply two numbers by constructing similar triangles. So for each of the three (x,y) points, I computed x^2 on the x-axis and xy on the y-axis (shown in red). Then I simply added up all the x^2's and xy's to get the green point in the top right which defines the OLS line.

$\endgroup$
2
  • 2
    $\begingroup$ To get the slope of the regression, I don't understand how you derived the point at the lower end as 0,0 ? It is not a requirement of a regression line to use 0,0 although it is true for many functions & your approach may be correct here too. Is there a generalised approach for calculating a second point that is independent of the first? $\endgroup$ – simon coleman Feb 2 at 9:42
  • 11
    $\begingroup$ @simoncoleman because it's simpler, I fit a model without intercept -- $y_i = mx_i + \epsilon_i$. Knowing that this is possible, it's pretty easy to convince myself (and hopefully everyone else) there's nothing which would make fitting an intercept impossible. i just didn't bother because i didn't think the added complexity would give any deep insight. $\endgroup$ – shimao Feb 2 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.