Closed form equations for simple linear regression estimators

Question

I'm learning specifically about different forms of simple linear regression including ordinary least squares, median absolute deviation, and Theil-Sen. I have no background whatsoever in linear algebra or even calculus. So I have a super hard time understanding even the notation or symbols of linear algebra.

I've learned that the ordinary least squares estimate for simple linear regression (meaning one independent and one dependent variable) have these simple equations for the slope and intercept (both evaluate to the same).

$$ \begin{aligned} a = r * \frac{sy}{sx} \end{aligned} $$

$$ \begin{aligned} a = \frac{\sum{(x_i - \bar{x})(y_i-\bar{y})}}{\sum{(x_i-\bar{x}})^2} \end{aligned} $$

In the first equation, a is the slope and r is the correlation and sy and sx are the standard deviation of y and x, respectively.

After calculating the slope, I can find the intercept i with this equation: $$ \begin{aligned} i = \bar{y} - r* \frac{sy}{sx} * \bar{x} \end{aligned} $$

My question is: Is there a nice closed form equation for the slope and intercept for Theil-Sen or median absolute deviation estimators? Is it the case there there just happens to be no closed form equation or that, in the case of Theil-Sen, there would never be a closed form equation because the estimator is non-parametric?

Answer 1

Extended comment about ordinary least-squares regression:

I don't think the single-variable case with slope and intercept is as easy to understand as the general linear model with a little linear algebra. Here we append 1s to data $X$ and a bias term in parameters to estimate $\beta$ so we can write in nice form

$$Y = X \beta + \epsilon$$

It turns out we can minimize least-squares error $\lVert Y - X \beta \rVert^2 $ with a unique solution if $X$ does not have colinearity, so that the columns of $X$ are linearly independent. The solution is

$$\beta = X^+ Y $$ where $X^+ = (X^T X)^{-1} X^T $ is the pseudoinverse of $X$.

The median absolute deviation cannot be written linearly so nicely.