# Closed form equations for simple linear regression estimators

I'm learning specifically about different forms of simple linear regression including ordinary least squares, median absolute deviation, and Theil-Sen. I have no background whatsoever in linear algebra or even calculus. So I have a super hard time understanding even the notation or symbols of linear algebra.

I've learned that the ordinary least squares estimate for simple linear regression (meaning one independent and one dependent variable) have these simple equations for the slope and intercept (both evaluate to the same).

\begin{aligned} a = r * \frac{sy}{sx} \end{aligned}

\begin{aligned} a = \frac{\sum{(x_i - \bar{x})(y_i-\bar{y})}}{\sum{(x_i-\bar{x}})^2} \end{aligned}

In the first equation, a is the slope and r is the correlation and sy and sx are the standard deviation of y and x, respectively.

After calculating the slope, I can find the intercept i with this equation: \begin{aligned} i = \bar{y} - r* \frac{sy}{sx} * \bar{x} \end{aligned}

My question is: Is there a nice closed form equation for the slope and intercept for Theil-Sen or median absolute deviation estimators? Is it the case there there just happens to be no closed form equation or that, in the case of Theil-Sen, there would never be a closed form equation because the estimator is non-parametric?

• "Non-parametric" does not automatically imply "no closed form:" it all depends on the formula. But since Theil-Sen requires finding a median based on slopes of all distinct point pairs, it is intuitively unlikely that any simplification exists--and if one did exist, finding it would be a matter of developing a clever algorithm rather than performing an analytical mathematical calculation.
– whuber
Commented Apr 6, 2022 at 13:47
• That's a good distinction - I was thinking that non-parametric implied no closed form, but I'm seeing now that this wrong Commented Apr 6, 2022 at 15:07
• Why do you need a closed form in the first place? Commented May 28 at 15:54

I don't think the single-variable case with slope and intercept is as easy to understand as the general linear model with a little linear algebra. Here we append 1s to data $$X$$ and a bias term in parameters to estimate $$\beta$$ so we can write in nice form
$$Y = X \beta + \epsilon$$
It turns out we can minimize least-squares error $$\lVert Y - X \beta \rVert^2$$ with a unique solution if $$X$$ does not have colinearity, so that the columns of $$X$$ are linearly independent. The solution is
$$\beta = X^+ Y$$ where $$X^+ = (X^T X)^{-1} X^T$$ is the pseudoinverse of $$X$$.