In predictive modeling, unbiased models can have higher variance, & thus be less accurate. Modelers may prefer some bias to maximize accuracy. Use this tag also for questions about the bias-variance decomposition.

243 questions
Filter by
Sorted by
Tagged with
38 views

### Practical usage of the bias variance tradeoff

I understand the bias-variance tradeoff. But, I have never come across a scenario where that has changed anything in the modelling process. Is there any practical scenario that you have encountered ...
10 views

### What is the relationship between bias-variance and sensitivity-specificity for novelty detection?

An over or under-parameterized binary classification model (- vs +) tends to over or under-fit (bias-variance tradeoff). This leads to errors during prediction on unseen data. Depending on if ...
1 vote
45 views

### Understanding the computation of sample bias and variance

I believe I am confused in some fundamental way about the bias-variance tradeoff and I am trying to clear up my confusion. Sorry for a bit of a preliminary rambling -- I wanted to put my ...
45 views

### Prediction intervals and bias-variance tradeoff

I was looking for literature which connects prediction intervals with the bias-variance trade-off. Obviously both concepts deal with describing a mean squared deviation: the bias variance tradeoff ...
45 views

104 views

### Proving upper bound for truncated difference

Let $X$ and $Y$ be real valued random variables. And define a truncation operator as: \begin{align} X(\tau) = (|X| \wedge \tau) \; \text{sign}(X), \quad \tau > 0 \end{align} Now, I am not ...
1 vote
29 views

### Bias and variance for quantile estimates

Is bias variance tradeoff a thing for quartile regression? Can I assume the error for quantile estimation follows a certain distribution (e.g., estimated quantile - true quantile follows normal ...
1 vote
104 views

64 views

### What is fixed and what varies in the bias-variance decomposition?

I am reading about the bias-variance decomposition from An Introduction to Statistical Learning with Applications in R (Second edition at page 34). It states that $$Y = f(X) + \epsilon$$ where the ...
335 views

### Philosophical insight of Bias Variance Decomposition

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below: \operatorname{MSE}(\hat{\theta}) = \operatorname{tr}(\operatorname{...
52 views

### Why do you overfit if you train a linear regression model on a dataset that doesn't have enough datapoints?

First of all, definitionally speaking, linear regressions tend to underfit (have high bias, low variance). Additionally, just intuitively speaking, it seems like a linear regression would underfit in ...
25 views

### Sampling distribution, bias and variance of cross-validation methods (particularly LOOCV)

(TL;DR version below) If my understanding is correct, bias/variance are measures of goodness of fit of a statistical estimator w.r.t. the sampling distribution. So if I have a statistic $t(X)$ that ...
81 views

2k views

### Why do we say that the model has a high variance when variance is actually the measure of spread of the data and not some property of the model?

I am trying to understand the difference between bias-variance and overfitting-underfitting. If a modal overfits the data it means that it will not generalize well on new data because it over learns ...
1k views

### Poor model performance on certain out-of-sample data

We're noticing poor model performance on certain out of sample products. We have trained a ML model on about 2000 different products in a few markets. Our predictors include a) product ...
124 views

### Bias-variance trade-off in linear regression [closed]

As it’s understood, in the bias-variance trade-off, variance refers to overfitting of the model and it examines the variability of output predictions. Suppose we have a simple dataset with one ...
1 vote
25 views

### Variance analysis on boosting approachs, Is there any guarantee that boosting will not worse the weak learner variance or even get it better?

I'm looking for a theorical justification why boosting does work in pratice, I'm almost sure that this reduces the bias of their weak learners (assuming all weak learners have the same bias), but I ...
42 views

### Checking understanding about a derivation of bias and variance in the context of generalization in course notes

Sorry for the long image. This derivation of bias and variance was given in publicly available course notes (here) on pages 3 and 4. I understand the first derivation. They showed that y* was the best ...
389 views

I have the following question about the theoretical advantages vs. the empirical advantages of regularization (i.e. shrinkage). As far as I understand, this is the general idea behind regularization: ...
277 views

### Apart from the Bias-Variance "Decomposition" - is there a Bias-Variance "Proof"?

I am sure at some point, many of us have come across the "Bias-Variance Tradeoff" : The "error" of any "estimator" (e.g an estimator can be considered as a linear ...
967 views

### How does repeated k-fold cross validation identify model instability?

In these threads 1,2,3, cbeleites mentions that in a single k-fold cross validation you cannot tell whether the variance is caused by model instability or using a different test set. Hence, one can ...
389 views

### Bias-variance trade-off between LDA and QDA w.r.t. dimensionality

Consider the bias-variance trade-off between linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). Switching from QDA to LDA will generally yield a reduction in variance. The ...
1 vote
65 views

### Bias vs. variance

I have a question about bias/variance trade-off for different competing models. Say one has estimated model A and model B and calculated their respective train and test error. How does one yield an ...
1 vote