How can I look for correlations between variables with large deviations?

I'm researching the correlation between the magnitude (a measure of brightness) and redshift ($z$ - a measure of distance) for a variety of galaxies called quasars. Plotting the magnitude against $log(z)$ for one set of data looks like this: and this pattern is similar for all of the sets of data I'm looking at (black/blue plot same set as above):

These plots are for only a small set of the data, but I expect them all to follow roughly the same pattern; fairly closely packed at low redshift, and much more scattered at high redshift.

I've been trying to fit a straight line to the data, so that I can use the magnitude of a quasar to estimate its redshift. While the correlations are highly significant, the $r$ and $r^2$ values are far too low (only around 0.5 for some of the best fits) to lead to a useful estimate, shown here for the black/blue plots above: I'm looking at the problem as a physicist, but since my knowledge of statistics is poor, I'm wondering if there's a better way to determine a tight correlation between the magnitude and $log(z)$ that I'm missing.

Is there a statistical way to tighten up the regression coefficient?

• Could you add a quick list of references to describe your variables? Just the regression equation, and then references ($y =$ blah blah blah, $z =$ bleh bleh bleh). – ERT Aug 9 '18 at 22:57
• Do you mean like in the legend of the second graph? – Jim421616 Aug 9 '18 at 23:11
• Oh, I understand now. You wrote "redshift (z - a measure of distance)". I thought this meant $z$ and "a measure of distance" were two variables being differenced. You meant "redshift ($z$, a measure of distance)" – ERT Aug 9 '18 at 23:13
• So you are looking for some relationship $M \sim \alpha + \beta_i \cdot z$? In this case, $M =$ magnitude. – ERT Aug 9 '18 at 23:17
• Yes, exactly. That way, we can estimate $z$ just from the brightness, without requiring to look at the atomic frequencies in the light. – Jim421616 Aug 9 '18 at 23:19

Based on this Quora answer, you may want to try something with a bit more complexity. $M \sim \alpha + \beta_1 z + \beta_2 z^2$ is a suggestion in this article, though I personally wouldn't go this route right away. You should check your residuals to see if you are missing some higher-order relationship. This could indicate "A missing higher-order term of a variable in the model to explain the curvature." Since you only have one predictor, finding which explanatory variable is missing a higher term would be pretty easy. Unfortunately, a pattern in your residuals could also indicate a missing explanatory variable. This is why I would not advise jumping to this method without some further checking.

Unfortunately, since you are only looking at two dimensions (magnitude, redshift), there is little you can do. This is pretty helpful for your specific case, I believe. A quote from the article:

In some cases, it’s possible that additional predictors can increase the true explanatory power of the model. However, in other cases, the data contain an inherently higher amount of unexplainable variability. For example, many psychology studies have R-squared values less that 50% because people are fairly unpredictable.

Without additional predictor variables to add dimensionality, it would be difficult to increase your $R^2$. Here is a link to an article discussing the significance of $R^2$. According to the author,

Just because effect size is small doesn’t mean it’s bad, unworthy of being interpreted, or useless. It’s just small. Even small effect sizes can have scientific or clinical significance. It depends on your field.

If the only point of the model was prediction, my client’s model would do a pretty bad job.

But it wasn’t. The point was to see if there was a small, but reliable relationship. And there was.

If you were looking to draw some sort of conclusion from the aggregate data, you could do this with a low $R^2$. Prediction could prove problematic, though. One last quote, this time from a different statistics blog, explaining that you may be able to draw meaningful conclusions from your model, just not predict very well:

A low R-squared is most problematic when you want to produce predictions that are reasonably precise (have a small enough prediction interval). How high should the R-squared be for prediction? Well, that depends on your requirements for the width of a prediction interval and how much variability is present in your data. While a high R-squared is required for precise predictions, it’s not sufficient by itself.