I have a weighted and an unweighted linear regression (same dataset) and need a measure of how well one model can explain variation relative to the other (i.e. are the R-squared values significantly different?)

I have the R-squared value for both models. I can bootstrap a confidence interval for the R-squared values and perform a one-way anova on the two resulting distrubtions, but this is computationally intensive compared to a simple test. I will be performing this calculation for millions of instances, so every byte adds up.

A little context: I'm in bioinformatics measuring gene-gene coexpression with a Pearson correlation coefficient. I am asking whether some samples are contributing to a high coexpression correlation coefficient more than others, thus I am weighting out some samples and asking if this model can now explain variation better (or worse) than the unweighted model.

  • 1
    $\begingroup$ Why $R^2$? Moreover, if weights play part, why would you compare to unweighted variance? $\endgroup$
    – Tim
    Dec 16 '19 at 17:50
  • $\begingroup$ @Tim I'm interested in the goodness of the fit. The null hypothesis is that a defined subset of data points have no effect on the goodness of the fit. The working hypothesis is that, in fact, they do, and weighting them out will increase the goodness of the fit. $\endgroup$
    – zdebruine
    Dec 16 '19 at 19:06
  • $\begingroup$ @Tim but your question motivates a second thought. Perhaps I could do a weighted t-test on the residual least squares... if that makes any sense. $\endgroup$
    – zdebruine
    Dec 16 '19 at 20:07

I'd still appreciate any educated statistical advice. After spending a whole day on this, I finally decided to analyze residual least squares. I'm trying to show whether geneA regulates coexpression of geneB with geneC. So in a scatterplot of geneB expression (x) vs. geneC expression(y), where values contain different amounts of geneA (weights_x):

  1. Calculate residual least squares for all (geneA,geneB) points based on y = mx+b best fit line
  2. Normalize these values to x for every value
  3. Perform a two-tailed t-test on the same dataset, but in one of the cases multiply by apply weights_x.
  4. Calculate the sign of the p-value based on whether or not GeneA positively or negatively affects GeneB coexpression with GeneX.

I've got some preliminary data and this model seems to accurately recapitulate biological observations and expectations. It is much more scalable than bootstrapping and returns a nice, clear-cut statistical value.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.