9
$\begingroup$

Suppose that we want to know how the price of a house changes per meter square of the area of the house. Further suppose that I have a dataset as the following:

Area | Price
100    200k
120    230k
...

That is, only the area and the price of a set of houses.

Given this setup I can think of two different ways:

  1. Fit a linear model (using linear regression) and look at the coefficient of Area.
  2. For all pairs of houses $i$ and $j$ find $\frac{price_i - price_j}{area_i - area_j}$ then take the average.

My question: Are these solutions different? If yes, in what ways they are different? or pros and cons?

$\endgroup$
3
  • 8
    $\begingroup$ To see why they are profoundly different, consider what happens to your estimate when two houses in your dataset have the same area. $\endgroup$
    – whuber
    Apr 10, 2023 at 16:24
  • $\begingroup$ Oh yes, in that case we will have a division by zero, interesting, it seemed to me very intuitive. Is there a general name for the modeling error that I've made? $\endgroup$
    – Sanyo Mn
    Apr 10, 2023 at 16:36
  • 5
    $\begingroup$ "Inadmissible" is a technical term used for statistical procedures with an expected risk that is never better than some other procedure, no matter what the underlying probability distribution might be. Under most probability assumptions where ordinary least squares might apply, I believe your method would be inadmissible and the proof is to compare its performance to OLS. However, your method is closely related to a standard robust, nonparametric technique, called the Theil-Sen estimator. $\endgroup$
    – whuber
    Apr 10, 2023 at 19:08

2 Answers 2

11
$\begingroup$

DIFFERENT

We just need one example of these two being different to show that the two need not give the same result, so let's simulate an example.

x <- c(1, 2, -3)
y <- c(1, 6, 3)

# Fit a linear model to calculate the OLS slope coefficient
#
L <- lm(y ~ x)

# Find the pairwise slopes
#
slopes <- rep(NA, 3)
slopes[1] <- (y[2] - y[1])/(x[2] - x[1])
slopes[2] <- (y[3] - y[1])/(x[3] - x[1])
slopes[3] <- (y[3] - y[2])/(x[3] - x[2])

# Compare the OLS estimate with the mean of the pairwise slopes
#
summary(L)$coef[2, 1]
mean(slopes)

The OLS slope estimate is $0.2857143$, while the mean pairwise slope is $1.7$, so the two methods do not have to agree.

For a really interesting example, as is pointed out in the comments, consider what happens when two distinct $y$-values correspond to the same $x$-value. Will that code even run for x <- c(1, 2, 1)? Should it run?

A similar regression method that might be of interest is the Theil–Sen estimator (median pairwise slope instead of mean). In the above example, the Theil-Sen slope estimate is $0.6$, different from both the OLS estimate and the mean of the pairwise slopes.

$\endgroup$
1
  • 2
    $\begingroup$ thanks, I almost reinvented this estimator :), with the exception of using mean instead of median. $\endgroup$
    – Sanyo Mn
    Apr 10, 2023 at 17:35
8
$\begingroup$

What is equivalent to OLS is a weighted mean pairwise slope. Suppose you want to fit $Y=\alpha+\beta X$ and you have $n(n-1)$ pairs $(x_i,y_i,x_j,y_j)$ with slopes $\beta_{ij}$. The information about $\beta$ in a pair is proportional to $(x_i-x_j)^2$, and if you write $w_{ij}=(x_i-x_j)^2$, the OLS estimator is

$$\hat\beta_{OLS}= \frac{\sum_{i\neq j} w_{ij}\beta_{ij}}{\sum_{i\neq j} w_{ij}}$$ (where you use the convention $w_{ij}\beta_{ij}=0$ if $x_i=x_j$)

The proof involves relating the numerator to the U-statistic estimator of covariance $$\mathrm{cov}[X,Y]=\frac{1}{n(n-1)}\sum_{i\neq j} (x_i-x_j)(y_i-y_j)$$ and the denominator to the U-statistic estimator of the variance $$\mathrm{var}[X]=\frac{1}{n(n-1)}\sum_{i\neq j} (x_i-x_j)(x_i-x_j).$$

(To generalise to two predictors you take triples of points and so on. The algebra becomes more tedious and you need to know some formulas for determinants.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.