Linked Questions

0
votes
0answers
18 views

Would you please give some examples on Bias-Variance Trade off? [duplicate]

I am a new learner for Machine Learning and are confused about the idea of bias-variance trade-off. Would you please offer some specific examples or situations that a bias-variance trade-off occurs? ...
69
votes
5answers
15k views

What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
15
votes
4answers
1k views

What does interpolating the training set actually mean?

I just read this article: Understanding Deep Learning (Still) Requires Rethinking Generalization In section 6.1 I stumbled upon the following sentence Specifically, in the overparameterized regime ...
16
votes
4answers
19k views

Number of parameters in an artificial neural network for AIC

How can I calculate the number of parameters in an artificial neural network in order to calculate its AIC?
10
votes
4answers
3k views

How does one explain what an unbiased estimator is to a layperson?

Suppose $\hat{\theta}$ is an unbiased estimator for $\theta$. Then of course, $\mathbb{E}[\hat{\theta} \mid \theta] = \theta$. How does one explain this to a layperson? In the past, what I have said ...
14
votes
2answers
5k views

Biased bootstrap: is it okay to center the CI around the observed statistic?

This is similar to Bootstrap: estimate is outside of confidence interval I have some data that represents counts of genotypes in a population. I want to estimate genetic diversity using Shannon's ...
20
votes
2answers
4k views

Is there a graphical representation of bias-variance tradeoff in linear regression?

I am suffering from a blackout. I was presented the following picture to showcase the bias-variance tradeoff in the context of linear regression: I can see that none of the two models is a good fit - ...
8
votes
4answers
3k views

Why must one trade off between bias and variance?

Apparently, a learning algorithm must make a trade off between bias and variance when producing a hypothesis. Bias means systematic deviation from data. Variance refers to the error due to ...
13
votes
2answers
956 views

What if there is no true data-generating process?

I've been engaging in a number of forecasting efforts recently, and have rediscovered a well-known truth: That combinations of different forecasts are generally better than the forecasts themselves. ...
10
votes
2answers
3k views

Bias / variance tradeoff math

I understand the matter in the underfitting / overfitting terms but I still struggle to grasp the exact math behind it. I've checked several sources (here, here, here, here and here) but I still don't ...
11
votes
1answer
1k views

Modern machine learning and the bias-variance trade-off

I stumbled upon the following paper Reconciling modern machine learning practice and the bias-variance trade-off and do not completely understand how they justify the double descent risk curve (see ...
9
votes
1answer
550 views

When will a less true model predict better than a truer model?

In "To Explain or to Predict?", Pr. Galit Shmueli said that sometimes a less true model can predict better than a truer model. Why is it so? When will it happen? How does it happen? Is explanation a ...
2
votes
1answer
4k views

Interpretation of low bias and variance for train/test errors

Based on the extensive discussion in this post, I understand that the goal is to achieve low bias and low variance. Now, in terms of train and test errors, does it imply that achieving low bias and ...
3
votes
1answer
2k views

Relation between MSE and Bias-Variance

If MSE is $$ \mathrm{MSE}(\hat Y) = \mathrm{Var}(\hat Y) + \mathrm{Bias}^2(\hat Y) $$ and the Bias-Variance decomposition is given by $$ \mathrm{Err}(\hat Y\,|\,X=x_0) = \mathrm{Var}(\hat Y) + \...
0
votes
3answers
748 views

Why does machine learning work for high-dimensional data($n \ll p$)?

Consider the high dimensional data with which the number of features $p$ is much larger than the number of observations $n$. Machine learning algorithm is trained with the data. My first thought is ...

15 30 50 per page