# Linked Questions

4answers
13k views

### What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
4answers
2k views

### How does one explain what an unbiased estimator is to a layperson?

Suppose $\hat{\theta}$ is an unbiased estimator for $\theta$. Then of course, $\mathbb{E}[\hat{\theta} \mid \theta] = \theta$. How does one explain this to a layperson? In the past, what I have said ...
2answers
3k views

### Is there a graphical representation of bias-variance tradeoff in linear regression?

I am suffering from a blackout. I was presented the following picture to showcase the bias-variance tradeoff in the context of linear regression: I can see that none of the two models is a good fit - ...
4answers
2k views

### Why must one trade off between bias and variance?

Apparently, a learning algorithm must make a trade off between bias and variance when producing a hypothesis. Bias means systematic deviation from data. Variance refers to the error due to ...
2answers
2k views

### Biased bootstrap: is it okay to center the CI around the observed statistic?

This is similar to Bootstrap: estimate is outside of confidence interval I have some data that represents counts of genotypes in a population. I want to estimate genetic diversity using Shannon's ...
2answers
1k views

### Bias / variance tradeoff math

I understand the matter in the underfitting / overfitting terms but I still struggle to grasp the exact math behind it. I've checked several sources (here, here, here, here and here) but I still don't ...
1answer
423 views

### When will a less true model predict better than a truer model?

In "To Explain or to Predict?", Pr. Galit Shmueli said that sometimes a less true model can predict better than a truer model. Why is it so? When will it happen? How does it happen? Is explanation a ...
1answer
2k views

### Interpretation of low bias and variance for train/test errors

Based on the extensive discussion in this post, I understand that the goal is to achieve low bias and low variance. Now, in terms of train and test errors, does it imply that achieving low bias and ...
1answer
687 views

### Relation between MSE and Bias-Variance

If MSE is $$\mathrm{MSE}(\hat Y) = \mathrm{Var}(\hat Y) + \mathrm{Bias}^2(\hat Y)$$ and the Bias-Variance decomposition is given by  \mathrm{Err}(\hat Y\,|\,X=x_0) = \mathrm{Var}(\hat Y) + \...
0answers
715 views

### Bias Variance tradeoff from a Bayesian perspective

I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
1answer
163 views

### What is parameter instability? How can I measure it? [closed]

What is parameter instability and how can I measure it? If my model is having a hard time to forecast out-of-time samples, could parameter instability or populational instability be the cause of it?
2answers
78 views

### Total Cost Shrinkage

I have a question regarding Shrinkage Methods. I am currently writing a term paper about ridge regression and lasso and before explaining the two methods, I want to give some theory on why shrinking ...
1answer
73 views

### Improving a boosted regression model or change?

I am looking at a data set that contains multiple predictors and a continuous response. Using dismo along with gbm I built (a terrible one?) model. Using the package sROC, I got an AUC or 0.48 - so my ...