Linked Questions

61 votes
7 answers

Intuitive explanation of the bias-variance tradeoff?

I am looking for an intuitive explanation of the bias-variance tradeoff, both in general and specifically in the context of linear regression.
NPE's user avatar
  • 5,491
71 votes
5 answers

What problem do shrinkage methods solve?

The holiday season has given me the opportunity to curl up next to the fire with The Elements of Statistical Learning. Coming from a (frequentist) econometrics perspective, I'm having trouble grasping ...
Charlie's user avatar
  • 13.9k
13 votes
3 answers

How can you handle unstable $\beta$ estimates in linear regression with high multi-collinearity without throwing out variables?

Beta stability in linear regression with high multi-collinearity? Let's say in a linear regression, the variables $x_1$ and $x_2$ has high multi-collinearity (correlation is around 0.9). We are ...
Luna's user avatar
  • 2,335
8 votes
4 answers

Is MSE decreasing with increasing number of explanatory variables?

I am wondering, if there is a negative correlation between Mean Squared Error \begin{equation} MSE = \frac{1}{n} \sum (\hat{Y}_i - Y_i)^2 \end{equation} and the number of explanatory variables. ...
Joachim Schork's user avatar
20 votes
2 answers

Is there a graphical representation of bias-variance tradeoff in linear regression?

I am suffering from a blackout. I was presented the following picture to showcase the bias-variance tradeoff in the context of linear regression: I can see that none of the two models is a good fit - ...
blubb's user avatar
  • 2,580
8 votes
4 answers

Why must one trade off between bias and variance?

Apparently, a learning algorithm must make a trade off between bias and variance when producing a hypothesis. Bias means systematic deviation from data. Variance refers to the error due to ...
David Faux's user avatar
1 vote
1 answer

When should you check if assumptions are met when using stepwise selection?

Suppose I want to find a linear model with Gaussian error for a given data set. (The data set contains insurance claims and the end goal is to predict claim cost from claim features.) Also, suppose ...
lalessandro's user avatar
1 vote
1 answer

How to improve this logistic regression model

I am using following data and self-explanatory code to create a model for prediction of 'low' (low birth weight) from modified birthwt dataset. I am using 80% for training and 20% for testing: ...
rnso's user avatar
  • 9,517
9 votes
1 answer

About the meaning of ARMA parameters

I suppose that the main scope of an econometric models should be predictive or causal inference. Following this perspective it was shown that underspecified model can perform better than the correct ...
markowitz's user avatar
  • 5,089
15 votes
0 answers

Practical thoughts on explanatory vs predictive modeling [duplicate]

Possible Duplicate: Practical thoughts on explanatory vs. predictive modeling This question has been bugging me for some time, and I was going to write a blog post about it. However, I think it ...
James's user avatar
  • 151
2 votes
3 answers

Choice of a less true model over the truer model if it predicts better and my purpose is prediction

Sometimes a less true model predicts better than a truer model (When will a less true model predict better than a truer model?). So should I choose a less true model over the truer model if it ...
KuJ's user avatar
  • 1,566