# How to deal with non-uniform data density in regression?

I have a program that can detect an object in an image. It will either output a confidence in [0, 1], or "no detection" (which can be interpreted as 0 confidence). I have collected a bunch of images where the distance to the object varies.

The problem I'm trying to solve is to fit a curve to the expected detection confidence, given the distance to the object -- a regression problem. Now, there's the issue that the data I have collected does not cover all distances equally -- I have several "hot spots", distances where I have a lot of images, relatively.

1. Is it necessary to normalize the data density for regression? I would say yes.
2. How should I go about dealing with the issue?

What I tried so far is to create a histogram of distances, and divide the confidence by the nearest histogram value. However, there is too much noise for distances which are rarely observed (histogram value close to 0). Also, the discontinuities of the histogram carry over to the adjusted data.

What I am considering to do is to do some resampling of the data to achieve a more uniform spread. Any other ideas?

Here is a screenshot that shows, from left to right, the original data (confidence vs. distance), the histogram of distances, and the the normalized data (confidence divided by histogram value for that distance vs. distance):

• Your question is not problem. The frequency of covariates has no relation with bias (bias in statistical meaning). For example, $Y$ being weight, and $X$ being height of a person. The proportion of people in your sample with height >5.8` would have no relation with bias of the estimate of regression coefficients (intercept and slop). But this frequency has effect on the precision. Good design will give you higher precision of the estimate comparing with bad design giving the sample size are the same. You can refer to regression book to check the requirements of linear regression. May 29, 2017 at 23:11