Is there a goodness of fit metric similar to R^2 that can be used to evaluate a weighted regression? In my particular case, my data points are cities and I weight a regression of this data by city population. My concern is that the R^2 value does not take into account the weight (population) of the data (cities) so it views a residual on New York City the same as a residual on Cincinnati.

  • 1
    $\begingroup$ You may want to consider transforming your data pre-regression, normalizing whatever data you have by a proportional population metric. It is difficult to determine without more information about your data set. $\endgroup$ – ERT Aug 2 '18 at 15:15
  • 1
    $\begingroup$ The outcome metric is a quantity per 1000 residents and the independent variables are characteristics of each city such as median rent, unemployment rate, etc. I would like to keep the regression in these terms while weighting on city population and I am looking for a metric similar to R^2 that will account for the weighting in the data. Does that help? $\endgroup$ – cambonator Aug 2 '18 at 15:29
  • $\begingroup$ There are different ways of looking at this, and one of them is: you have created a mathematical model, and now want to run data through that model. To create the model you weighted data. If you run data through the model for a city that was not part of the regression, will you have the weights for that city? If the answer to that question is no, consider unweighted fit statistics to give you a clearer understanding of how the model will function on new city data where you do not have weights. $\endgroup$ – James Phillips Aug 2 '18 at 15:33
  • $\begingroup$ Hi James, since the weight is the population of the city, I think I would always have that information for any city I put into the model. $\endgroup$ – cambonator Aug 2 '18 at 15:42
  • 1
    $\begingroup$ Why not use population as an additional covariate? $\endgroup$ – ERT Aug 2 '18 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.