How do I assign weight to each observation of the data in weighted least square fitting? Suppose I have  a dataset { p1, p2,...pN}, where pi=(xi,yi), i=1,2,..., N.
How do I assign weight wi to each data point pi in weighted least square fitting?
Could any one help me? I'm not statistician. I have  just little knowledge about Statistics.
 A: Assume that the basic model is 
$$y_i=\mathbf x_i^T \boldsymbol \beta+\epsilon_i,$$
then weighted least squares can be written in the transformation
$$\sqrt{w_i}y_i=\sqrt{w_i}\mathbf x_i^T \boldsymbol \beta+\sqrt{w_i}\epsilon_i,$$
i.e., we regress $\sqrt{w_i}y_i$ on $\sqrt{w_i}\mathbf x_i$. Thus we have the weighted sum of squared residuals,
$$\sum_{i=1}^{n} w_{i}(y_i-\mathbf x_i^T \boldsymbol \beta)^2.$$
The objective is to minimize the weighted sum of squared residuals. Therefore, the estimates of the parameters are the solution for the following modified normal equations,
$$\mathbf{\left(X^TWX\right)\hat {\boldsymbol {\beta}}=X^TWy},$$
where $\mathbf{W}=\mathrm{diag}(w_1,w_2,\ldots,w_n)$. We usually use the inverse of the variance of error term as the weights, $$w_i\propto\mathrm{var}(\epsilon_i)^{-1}.$$ The idea is that the observation with higher variance would have lower weight. 


*

*If the weights (or variance structure) are known, say $\mathrm{var}(\epsilon_i)=\sigma^{2}x_i^2$ or $\mathrm{var}(\epsilon_i)=\sigma^{2}x_i$, we can directly use weights $w_i=x_i^{-2}$ or $w_i=x_i^{-1}$. 

*Unfortunately, for most cases, the weights (or variance structure) are unknown, we need to use one of the following methods:


*

*Use residual plot of the squared residuals against the predictor $x_i$ (or the fitted values $\hat y=\hat\beta_0+\hat\beta_1x_i$) to detect possible variance structure.

*Use formal test to detect the variance structure, see the Page 196 of the reference.

*Iteratively reweighted least squares (IRWLS) to model the variance structure at each iteration step, e.g. $\mathrm{var}(\epsilon_i)=\gamma_0+\gamma_1x_i$ by regressing $\hat\epsilon^2_i$ on $x_i$.

*Use the likelihood method to simultaneously estimate the mean parameters $\boldsymbol \beta$ and the variance parameters $\boldsymbol\gamma$.


A: I am not statistician either but, as a physicist, I encountered several times this situation and I tried to make my personal philosoply. So, please, just consider this answer as a very personal opinion.  
Let me consider a set of data points [X(i) , Y(i)] with no information about variance and say that we want to minimize a weighted sum of squares. If the range covered by the Y's is very large, the points with the largest values of Y are really those which define the regression. So, in such a case, what I minimize if the sum of the squares of relative errors on the Y's which then corresponds to a weight equal to [1 / Y(i)]^2. In such a situation, all points have the same importance. Otherwise, I do not weight my data.  
For sure, I could produce counter arguments to that : points with very low Y's for example.
