I am trying to carry out a simple linear regression analysis represented by the standard formulation: $$ y_i = \beta_0 + \beta_1x_i +\epsilon_i $$

However, there is an issue with my data. There is no guarantee that the response variable values are the true values and the probability of it being true decreases as the value of $y_i$ increases, i.e., the higher the value of $y_i$, the less we are certain that $y_i$ is the true value.

I am trying to deal with this using a weighted least square regression where I assign weights $1/y_i$ to each $y_i$ to represent the uncertainty of $y_i$.

There are certainly other ways of assigning weights but I am not sure if this kind of procedure makes any sense. Moreover, if the resulting model is significant, does it actually suggest there is a relationship between $x$ and $y$?

Thanks very much!

P.S. I think I need to provide a bit more of context. The data I have is derived, both $y$ and $x$ where $y$ is the nearest distance between two point patterns $A$ and $B$. From prior knowledge, $B$ is generated by $A$ but I do not know which point in $B$ is generated by which point in $A$. So I have to make assumption that the closer the point $B$ is to $A$ then point $B$ is generated by $A$. $x$ is some other quantity associated with $A$. So now the problem is that the farther away a certain point $B$ is from its nearest neighbor in $A$, the less certain I am that this point in $B$ is actually generated by that point in $A$. So in some way I am trying to penalize the validity of large values of $y$.

  • 1
    $\begingroup$ Try to fit simple linear regression, get the residuals, generate a scatterplot of residual vs fitted value. Check if the variation is increase along the fitted value. If yes, try to use weight, if not, maybe you thought that variance increase along the value of y is wrong. $\endgroup$ – user158565 Nov 21 '18 at 19:41
  • $\begingroup$ @user158565 Thanks for you comment. I added a bit of context to my problem. It seems what you are suggesting are heteroskedasticity in the data. But I don't think that is the same issue I have here. $\endgroup$ – davidolohowski Nov 21 '18 at 19:57
  • $\begingroup$ You want to study $y$ the nearest distance between two point patterns $A$ and $B$, or if $B$ is generated by $A$? $\endgroup$ – user158565 Nov 21 '18 at 20:12
  • $\begingroup$ @user158565 I know $B$ is generated by $A$ but I do not know which point in $B$ is generated by which point in $A$. What I want to study is the relationship between the distance from $B$ to its generator in $A$ and some features associated with that generator point in $A$. So I am assuming that a point in $B$ is generated by its nearest neighbor in $A$. But this assumption can be wrong when that nearest neighbor distance is too large. $\endgroup$ – davidolohowski Nov 21 '18 at 20:18
  • 1
    $\begingroup$ Then I think it is heteroskedasticity thing. But before you check the heteroskedasticity, you want to punish heteroskedasticity. $\endgroup$ – user158565 Nov 21 '18 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.