How to deal with (heteroscedastic) errors in random forest

Question

I am running a random forest on a dataset (only numerical) and achieve the following predictions:

Thanks to question and answers such as here, I understand why the predictions vs truth values don't lie around the y ~ x line.

I could stop here and accept this as the final model but I have a couple of questions around potentially improving it.

1. Is it reasonable to fit a linear model to the predictions vs truth to correct for the mismatch between these and the y ~ x line? Instinctively I think this is probably a bad idea, but I'm not entirely sure why. I've seen a blog that starts with a linear model and then fits an RF to the residuals - can't find it again now, sorry (Edit: found it) - perhaps that's a better way to go?
2. How best to deal with the heteroscedastic errors? I have already done all the transformations that seem reasonable - e.g. % variables to logit, log-normal for strictly positive variables near zero, obvious things like that. I don't think I need to worry about including interaction terms as I thought RFs would handle those automatically. Maybe there's something obvious I'm missing but, failing that, what's the best next step? I could combine this with question 1 using a glm I suppose?

I'm really at the limits of my modelling skills now and would really appreciate some pointers.

Edit: just to clarify - I care about improving the predictions not whether the errors are heteroscedastic or not per se. I'm just hoping if I can improve that then I can improve the noise, particularly at the low values, and get better predictions. But as @Tim noted, it probably more an issue of data quantity in that region than anything as there's no reason why reducing the heteroscedasticity ought to improve the predictions. That still leaves question 1 though - is it reasonable to regress on the fit?

Answer 1

• Start with the Why don't linear regression assumptions matter in machine learning? thread. Assumptions for statistical models are needed for inference, while with machine learning models we care about making accurate predictions.
• Even in the case of linear regression, there are misconceptions about its assumptions and their importance, as you can learn from the What is a complete list of the usual assumptions for linear regression? thread. Homoskedasticity in linear regression is needed for calculating confidence intervals and running hypothesis tests, not the things you care about or even are possible to do with random forest. If you don't care about those, you can even use linear regression if the assumption is not met.
• The homoskedasticity assumption comes from the distributional assumption about independent and identically distributed noise $\varepsilon_i \sim \mathcal{N}(0, \sigma^2)$. The random forest does not make any distributional assumptions.

So nothing is broken and nothing needs fixing. You may care about the predictions being biased and may want to do things like gathering more data (especially for the values <60 and >70), feature engineering, hyperparameter tuning, ensembling models, etc, to improve the predictions, but it does not have anything to do with homoskedasticity.