How to measure bias variance trade-off I got these results after fitting a model:

In the above picture X axis shows the fitted values and Y axis shows the truth values.
We can see bias although I tuned reg_alpha & reg_lambda (i used XGBoost and also tuned learning_rate, n_estimators, max_depth, min_child_weight, gamma=0.1, etc.) 
So how to measure bias? Is this small or large bias? Or can we use the model or try tuning the parameters again? 
 A: You can measure the bias-variance trade-off using k-fold cross validation and applying GridSearch on the parameters.
This way you can compare the score across the different tuning options that you specified and choose the model that achieve the higher test score.

I also encountered a useful reference about bias-variance trade-off. In the section 3.3 - Analytical Bias and Variance the author make an analogy with the error calculation formula of a kNN algorithm:
"In the case of k-Nearest Neighbors we can derive an explicit analytical expression for the total error as a summation of bias and variance:"

http://scott.fortmann-roe.com/docs/BiasVariance.html
A: The difference between your predictions and the real values is the absolute error (and not bias). This error is composed of the variance of your model, the bias of the model, and the irreducible error (that you cannot reduce). 
I guess that what you want to do is to measure the error and optimize your model to minimize this error. Look for a function in your toolkit to measure the error in your predictions (e.g. MSE). However, if you are already optimizing your model using cross-validation, you are probably getting closer to the best result (it depends on how you are searching your parameters). Maybe you could be more specific about how you are choosing these parameters?
A: I wrote an article on this exact subject here. This should give you some clues about how to tweak parameters such as learning rate, lambda, subsample and number of rounds. This is what a bias variance plot looks like for learning rate:

As others have mentioned, this is not an analysis you can typically do for real datasets as we have to know about how the data set was generated. Having said that, it useful to observe how these different xgboost parameters interact with a synthetic dataset to understand at a high level how they manipulate the model.
