Is there a good replacement of sum of squared deviations that do not tend to split on edges?

Question

I build a predictive model (regression) on a dataset that has just one real-valued feature and one real-valued target. To make it even simpler I want to find just a step function (decision tree with depth 1).

I try to use the sum of squared deviations as a measure of the split quality. I have notices that often the spit is very close to the ends of intervals. In other words, number of points on one part of split is much larger than the number of points in another part. For example 1995 and 5 for 2000 data points.

First, I though that this is an artefact of my data set (that targets for very small and very large values of features are special). However, when I run the same procedure on a random data set (where there is no dependency on feature per construction) the splits also have tendency to be closer to the ends.

Is it looks like a drawback of "vanilla" sum of squared deviations. Shouldn't we expect that for date with no dependency on feature the location of the optimal split should be uniformly distributed? Is there a measure that has this property?

ADDED 1:

In decision tree regression there is a "minimal leaf size" parameter that prevents the describe behaviour. However, I would like to have a more natural way to achieve what I need.

ADDED 2:

I took a closer look on this behaviour. It is not the case that splits closer to one of the ends have a tendency to give better results (smaller squared deviations). However, it is the case that the squared deviations for those splits have larger variation and, therefore, they have better chances to give the smallest squared deviation.

ADDED 3:

It looks like the variation of the split quality (sum of squares) for random data depends on distribution of targets. If targets distributed normally, there is no dependency on the location of the split. For my original distribution, as I have already mentioned, the variation was higher closer to the both ends of the split interval. For uniform distribution, however, the variation tends to be smaller on the ends of the interval.

ADDED 4:

If my description is not clear, here is a Python code that demonstrate what I mean for an uniform distribution:

import numpy as np
size = 30
X = np.arange(size)
all_Es = []
for exp_ind in range(100000):
    Y = np.random.uniform(size = size)
    Es = []
    for split_ind in range(1, size):
        Y1 = Y[:split_ind]
        Y2 = Y[split_ind:]
        m1 = np.mean(Y1)
        m2 = np.mean(Y2)
        P = np.where(X < split_ind, m1, m2)
        e = np.mean(np.power(Y - P, 2))
        Es.append(e)        
    all_Es.append(Es)
all_Es = np.array(all_Es)
V = np.std(all_Es, axis = 0)
import matplotlib.pyplot as plt
plt.plot(V, marker = 'o')

ADDED 5

Since there was a complain, that my text is not clear enough, here is another try.

1. I have a list of real values. For example: [1.2, 3.4, 5.6, 7.8, 9.0]
2. I split this list into two parts. For example: [1.2, 3.4] and [5.6, 7.8, 9.0]
3. For each split I use mean of the values as prediction.
4. I check the accuracy of the model using squared deviations.
5. I find a split that minimises the squared deviations.

Problem:

For not normal distribution of values in the original list, the location of the split either tend to be closer to the ends of the list (to the beginning or to the end), or it tends to be closer to the middle of the list. I observer one or another type of behaviour depending on the distribution.

Answer 1

Outliers have a strong influence on sums of squares. I suspect that if an outlier occurs close to the interval ends, according to sum of squares it will be good to have a split there, as this can eliminate the influence of that outlier from the sum of squares of the other points. If the outlier occurs somewhere in the middle, this is not the case, as regardless of where the split is, this outlier will still be together with a good number of other points. Now if you use heavy-tailed distributions, you can expect a couple of outliers in the data and in each interval that has enough points, some in the middle and the odd one close to the interval ends, and I suspect that in such cases the best job according to sum or squares is done "isolating" an outlier on the side together with few points. (You say that this does not happen with normal or uniform data, which makes sense as these hardly ever produce strong outliers; the uniform has no outlier-generating tail at all.)

I'd expect this to "improve" with L1- or Huber loss.