Linked Questions
12 questions linked to/from Can overfitting and underfitting occur simultaneously?
27
votes
4
answers
3k
views
Why is there an asymmetry between the training step and evaluation step?
It is well-known, especially in natural language processing, that machine learning should proceed in two steps, a training step and an evaluation step, and they should use different data. Why is this?...
- 373
20
votes
4
answers
2k
views
Is Fig 3.6 in Elements of Statistical Learning correct?
Here is the figure from the textbook:
It shows a decreasing relationship between subset size $k$ and mean squared error (MSE) of the true parameters, $\beta$ and the estimates $\hat{\beta}(k)$. ...
- 528
13
votes
2
answers
4k
views
Bias / variance tradeoff math
I understand the matter in the underfitting / overfitting terms but I still struggle to grasp the exact math behind it. I've checked several sources (here, here, here, here and here) but I still don't ...
- 311
13
votes
2
answers
29k
views
How to know if model is overfitting or underfitting?
I understand that using cross validation we can validate our model, but it is also possible that maybe our model is underfitting; hence, providing wrong results. One possibility that I can think of is ...
- 241
3
votes
2
answers
7k
views
Why the error on a training set is decreasing, while the error on the validation set is increasing?
When training XGboost model I observe the following outputs:
...
- 499
8
votes
1
answer
9k
views
Neural network over-fitting
I've learned that over-fitting can be detected by plotting the training error and the testing error versus the epochs. Like in:
I've been reading this blogpost where they say the neural network, net5 ...
- 1,185
6
votes
1
answer
17k
views
Is it possible for test error to be lower than training error
Is it possible to have test error lower than training error?
I have a classification problem with 2000 samples, 500 of which are positives, 1500 are negatives. I split my data into 70% training data, ...
- 63
9
votes
2
answers
539
views
What are differences between industrial statistics and social science statistics/econometrics that are potential stumbling blocks?
Main question up front: what are differences between econometrics/social science statistics that and industrial statistics that people switching between the two should be are of?
I got a PhD in ...
- 1,413
6
votes
3
answers
427
views
Impossible to overfit when the data generating process is deterministic?
For a stochastic data generating process (DGP)
$$
Y=f(X)+\varepsilon
$$
and a model producing a point prediction
$$
\hat{Y}=\hat{f}(X),
$$
the bias-variance decomposition is
\begin{align}
\text{Err}(...
- 69.5k
1
vote
1
answer
332
views
What are the reasons why a classifier could produce bad results?
I know of four possible reasons:
overfitting
underfitting
input data doesn't represent the problem (which I guess is underfitting)
classifier isn't suitable (e.g. problem is not linear)
Are there ...
- 807
1
vote
2
answers
92
views
Overfitting or under-fitting. which one is the most common error that happens in classification tasks?
I have read many blogpost and articles about overfitting and underfitting, and I have, to some extent, understood what they exactly are, and different ways to overcome these two problems.
However, I ...
- 111
3
votes
1
answer
94
views
Bias of a model that nests the DGP
Consider model 1 and model 2 where the former is a special case of the latter. E.g. model 1 is $y=\beta_0+\beta_1 x+u$ while model 2 is $y=\gamma_0+\gamma_1 x+\gamma_2 x^2+v$. Suppose model 1 is the ...
- 69.5k