1
$\begingroup$

I'm performing a weighted linear least squares fit, where the weights correspond to the number of counts of a specific observation. Due to the nature of the data, it is possible that a small handful of observations get weighted much more than the rest, so that the regression is dominated by, say, two data points only, which obviously leads to bad results.

I've run a few tests and I noticed that by e.g. capping the weights, I can improve the results, though this feels rather hack-ish to me. Are there better ways to avoid a small number of data points to out-weigh the rest of the data, or, if capping the weights is a good approach, is there non-ad hoc way to determine the optimal value of the cap?

EDIT

Here's what some sample data and fits look like with (a) no weights, and (b) weights. The data were generated from fairly realistic simulations, so I know the ground truth (red line).

enter image description here enter image description here

My problem is that the weight of some data points (#2 at x=10 for example) can be sufficiently large to dominate the fit. However, I also don't want the very-low-count data to weigh in too much, otherwise I get a really crappy fit as well.

$\endgroup$
  • 3
    $\begingroup$ If you don't like the weights, why are you using them? Capping the weights is esentially using a different weighting function, and is also suspect in the sense that you are in effect giving more weight to the infrequent events (typically, outliers) than to the most frequent events. "Improving" the results by modifying the weights also makes me wonder whether you have a predetermined result to arrive at in mind, and are trying to come up with a weighting function that will achieve the result that you want to get. Else, how do you know that changing the weights improves the results? $\endgroup$ – Dilip Sarwate Jan 11 '12 at 17:42
  • 4
    $\begingroup$ Is it really the case that a small handful of observations is getting weighted more heavily if the weight is the number of counts of the observation? From your description, it seems to me that each physical observation is really $n_i$ actual observations, in which case each actual observation is getting a weight of one (using your weighting scheme.) $\endgroup$ – jbowman Jan 11 '12 at 18:19
  • 1
    $\begingroup$ Is there something different about the frequently occurring observations that you could account for in the model? i.e. a reason why you may expect them to be "different" to the rest? If not, then you cannot reasonably claim that they are "skewing" the results. $\endgroup$ – probabilityislogic Jan 11 '12 at 20:40
  • 1
    $\begingroup$ @DilipSarwate: Since I'm testing this on simulated data, I know exactly which result I should expect, and I want to find out how I can get a trustworthy fit to data I know before I start working with the real data coming off the sensor. Also, you're right in that I'm using the weights to penalize the low-weight data, so making them more important is indeed an issue. $\endgroup$ – Jonas Jan 12 '12 at 17:13
  • 1
    $\begingroup$ For counts, several considerations suggest weighting the data inversely to the counts themselves. (1) Counts often have Binomial or Poisson-like distributions, (2) the variance of such a distribution is close to its expectation, and (3) optimal weights should be inversely proportional to variances. You need to adjust the weighting slightly for very low counts (you cannot divide by zero); one way is to add a small constant to all counts (around 1/6 to 1 generally work) before the division. $\endgroup$ – whuber Jan 16 '12 at 15:26
1
$\begingroup$

You can try applying a function to the weight. Reasonable choices would be a log or sigmoid.

$\endgroup$
  • $\begingroup$ Thanks for the suggestion. Log won't work in my case, because it gives too much weight for the low-count data, but sigmoid seems to be exactly what I was looking for. Is there any good rule of thumb for choosing the transition point? $\endgroup$ – Jonas Jan 16 '12 at 13:57
  • 1
    $\begingroup$ This sure sounds ad hoc to me. $\endgroup$ – whuber Jan 16 '12 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.