In predictive modeling, unbiased models can have higher variance, & thus be less accurate. Modelers may prefer some bias to maximize accuracy. Use this tag also for questions about the bias-variance decomposition.

97 questions
Filter by
Sorted by
Tagged with
46 views

### Bias-variance decomposition for time series

Suppose you have a time series $y_t$. What is the bias-variance decomposition of $$E[[y_{t+1} - \hat{y}_{t+1}]^2|y_1, \dots, y_t]$$ where $\hat{y}_{t+1}$ is some forecast of $y_t$. I tried to ...
78 views

### Bias-Variance Trade Off with Cauchy Estimator

I'm having a look at the bias and standard error of a set of estimators. I expected to see the trade off when varying the parameter of the estimator, but I see that both the bias and the variance ...
20 views

45 views

### What to do if recombination of independent variables cause multicollinearity issue?

Let's say you use a regression that has either: 1) interaction variables or 2) polynomials. When using those features you may run into multicollinearity issues. Do you know how to resolve this issue?...
34 views

### Estimate of population mean that minimises squared error

When reading Susan Athey's lecture slides here, I was confused by her claim that the population mean estimate for a given sample that minimises squared prediction error was not the sample mean, but ...
129 views

### Tradeoffs of robust mean measures (trimmed, Huber, cosh, etc)

After recently having delved into the world of robust measures (for location, mean being the classical case), I have had difficulty understanding robust measures' core dynamic. Basically, what are ...
723 views

### Why does bagging increase bias?

In machine learning, why does bagging increase bias? I've read that using less data would lead to a worse estimate of the parameters, but isn't the expected value of the parameter constant regardless ...
385 views

### multicollinearity resulting in high variance

Section 8.7.1 of Elements of Statistical Learning talks about high variance in a classification tree due to high correlation between features. What is the intuition behind this? I would think that ...
36 views

### What is the relation of bias in the Bias-Variance trade-off to underfitting

In machine learning literature, people often talk about the bias-variance trade-off as well as overfitting and underfitting in the same paragraph. However, in these contexts, bias and variance comes ...
91 views

### regression model can't achieve low bias and low variance at the same time

I am running a RandomForestRegressor model on a dataset and it seems it can NOT achieve low bias and low variance at the same time. So I suspected that the input (independent) variables are not ...
146 views

### Is there a theoretical reason why simple models perform better than complex models on time series forecasting tasks?

Empirically, simple forecasting methods such as damped trend exponential smoothing, STL, or even random walks typically outperform more complex models such as higher order ARIMA models or ML based ...