3
$\begingroup$

I am trying to fit a curve to set of data points. I have a set of weights that I want to assign each point while fitting. My question is that should i normalize the weights or use the weights as they are? Are there any pros and cons of following one approach over the other.

Pardon me if the question is all too basic and it would really help if you can point me to some literature.

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ If no weights are used, they are all implicitly equal to 1.0. If all wights are set equal to 2.0, then the same results should still be given as no individual data point has more or less weight than any other data point. So the weights are relative, for example if all weights are set to 0.1 except for a single data point where it is set to 0.2 should give the same results as setting all weights to 10, except for that same point having a weight of 20. Normalization of the weights should not change their relative size, so it should have no effect. $\endgroup$
    – James Phillips
    Nov 22, 2018 at 18:08
  • $\begingroup$ Thanks a lot James, but wouldnt that depend on the way way we normalize..if we do a simple average, then what you say is right, but otherwise there might be some difference, right? $\endgroup$
    – nimbus3000
    Nov 26, 2018 at 3:26
  • $\begingroup$ Well, if one of the normalized weights had a value of zero then the associated point is no longer used in the regression. So it does seem possible to normalize incorrectly. $\endgroup$
    – James Phillips
    Nov 26, 2018 at 15:26

1 Answer 1

Reset to default
1
$\begingroup$

Since you have not specified how your "weights" enter your particular model, it is not possible to definitively say what you can and can't do with them. However, in most models that use weights, the model is amenable to any weighting vector that has non-negative weights. It is usually possible to scale the weights to "normalise" them, but in most models this is not necessary. Nevertheless, it can make sense to normalise a weighting vector to make it comparable to the standard case where there is no weighting.

In a standard statistical analysis, all the data points have the same weighting in the analysis, which means that they all have an implicit weight of one. Thus, in a standard analysis with $n$ data points you could say that you have an implicit weighting vector $\mathbf{u} = (1, ..., 1)$. Since there are $n$ values, we have $||\mathbf{u}|| = \sqrt{n}$. Hence, if you want to normalise a weighting vector $\mathbf{w}$ to give the magnitude of the standard case you would usually set the normalised weight vector as:

$$\tilde{\mathbf{w}} \equiv \mathbf{w} \cdot \frac{\sqrt{n}}{||\mathbf{w}||} \quad \quad \quad \implies \quad \quad \quad ||\tilde{\mathbf{w}}|| = \sqrt{n}.$$

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks a lot for the answer Ben. The weight themselves are normally distributed and taken from another distribution - is this what you mean when you say how the weights enter the model? $\endgroup$
    – nimbus3000
    Nov 26, 2018 at 3:31
  • $\begingroup$ @nimbus3000: When you say that they are normally distributed, does this mean you have some negative weights? What does that mean exactly? $\endgroup$
    – Ben
    Nov 26, 2018 at 4:17
  • $\begingroup$ No, all positive weights....the weights are a measure of volatilities which are normally distributed...they cant be negative $\endgroup$
    – nimbus3000
    Nov 26, 2018 at 5:18
  • $\begingroup$ @nimbus3000: Just be careful here, because the normal distribution generates values over all the reals, including negative values. In any case, what I mean when I talk about weights "entering a model" is that in regression, GLMs, etc., there is usually provision for an allowance for weights on the observations. The models are usually constructed in a way that normalisation of the weights is unnecessary, but the above method gives a simple normalisation that can be used to obtain a fixed norm. $\endgroup$
    – Ben
    Nov 26, 2018 at 6:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.