Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

3
votes
0answers
31 views

Overfitting in Random Forest Classifier?

I would like some help from you in a classification model that I am developing. In summary, the problem is: – Classification problem with binary outcome (0/1) – The classifier is a Random Forest ...
0
votes
0answers
36 views

Emperical evidence if a model has too much parameters it will never overfit [on hold]

This Question stems from the comments on this answer, and also this answer, the premise that is presented by User Vlad is if you have way too much parameters, your model will not overfit (What comes ...
1
vote
1answer
26 views

Neural Networks - Difference between 1 and 2 layers?

I'm currently working on a regression problem, using neural networks to constrain parameters for a complex physical scenario. I am searching the hyperparameter space for the best model and have thus ...
0
votes
0answers
13 views

How can I fine tune simple RNN or LSTM? [on hold]

I'm dealing with RNN and LSTM models by normalized data in range of [-1,+1] and reshaped data for each time sequence from 3 individual matrices A,B,C to long row includes elements of all 3 matrices ...
0
votes
0answers
25 views

validation vs test vs training accuracy, which one to compare for claiming model overfitting?

I have read on the several answers here and on the internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which ...
0
votes
1answer
24 views

Bad performance on training set and increasing validation loss: Overfitting or underfitting?

From my understanding of over- and underfitting, the two behaviors are not completely mutually exclusive. Overfitting can pretty much always be achieved if the model has enough capacitance (is able to ...
2
votes
2answers
69 views

Is overfitting always a problem?

If I test various models and the best performing model also happens to be one that appears to be overfit, is this an issue? For example, if I have a model with 100% accuracy on the training data and ...
0
votes
1answer
37 views

How to detect if model is overfitting?

I know this question is asked billion times, but I could not really find an answer to my situation. So, I want to show all the logs of Keras model learning. The problem is I don't know if my model is ...
1
vote
1answer
11 views

Overfitting when train and test set features have identical distributions

In a hypothetical setup where train and test set features have identical distributions, is the correct to say that one can not overfit.
2
votes
0answers
28 views

Overfitting cross-validation scores

I've ran 1000 iterations of XGBClassifier parameters search using RandomizedSearchCV on the titanic dataset. That's just for context, but the question applies to any CV search method, any model and ...
1
vote
1answer
40 views

How to validate models beyond checking for overfitting

I have an unusual problem, which is that my model is performing too well and I am struggling to trust it. The data is a table of "snapshots" about moments in games of chess. For example, a game that ...
0
votes
0answers
17 views

How to prevent heteroskedastic models from overfitting?

I'm trying to fit neuroscience data using a Gaussian Process, but noticed that it behaves poisson-like (var = mean). Since classic GP models assume iid noise, I figured I could get a better fit by ...
1
vote
0answers
22 views

How can I avoid overfitting when training a model to predict the outcome of live events

I have a question about predicting the outcome of 'live' events - I.E. and event which has begun but not yet finished. Say your goal is to predict whether or not a particular player will win a series ...
0
votes
1answer
24 views

Training loss decreasing, Validation loss steady, where to stop?

In the following training scenario (Orange: training loss, Blue: validation loss), what epoch is the best time for stopping the training? Validation is almost steady as we continue training, but ...
19
votes
7answers
1k views

Overfitting and Underfitting

I have made some research about overfitting and underfitting, and I have understood what they exactly are, but I cannot find the reasons. What are the main reasons for overfitting and underfitting? ...
2
votes
1answer
53 views

What if cross-validation fails to prevent overfitting

I'm training a random forest model with AUC as performance metric. I've splitted my data to train set (70%) and test set (30%) and performed cross-validation on train set to tune the hyperparameters. ...
0
votes
0answers
42 views

Is this an example of overfitting?

I am trying to predict some future values using either KNN or regression model. I have about 9 independent variables that do not seem to have strong correlation to each other (Not completely sure ...
0
votes
1answer
15 views

How to prove High Sampling Variance in over-fitted functions

I've been reading recently about over-fitting and it is frequently related to High Sampling Variance and Low Bias characteristics. However, what is the metric used to state the High Sampling ...
0
votes
1answer
29 views

Random Forest feature selection over-fitting doubt

My objective is to find genes that can be used as biomarkers with low error. I am using Random Forest (RF) using R package randomForest and following the steps in below link as it is has similar ...
0
votes
0answers
20 views

Is it Overfitting if validation metric is improving but training metric is extremely high?

If Accuracy or AUC for 4 different values of a tuning parameter in a model, ntrees in randomforest from 500 , 1000 , 1500 , 2000 over Training Data is 0.7 , 0.8 , 0.9 , 1.0 over Test / Validation ...
0
votes
0answers
11 views

Wrong weights learned when training RBM

I'm training my RBM network and on epoch #4 I have such a filters representation (my weights matrix) But on the next iteration (fifth epoch) something went wrong and my filters became like this What ...
4
votes
2answers
145 views

Can Adjusted R squared be equal to 1?

I have a dataset with around 15 independent variables. I am using a multi-regression model to fit the dataset. For model selection, I am using a backward elimination procedure based on the p-values. ...
0
votes
1answer
87 views

How to distinguish overfitting and underfitting from the ROC AUC curve?

For model selection, one of the metric is (AUC Area Under Curve) which tell us how the models are performing and based on AUC value we can choose the best model. But how to distinguish whether a ...
0
votes
0answers
37 views

do not need many controls with big data?

I am looking at a paper which uses a large panel data, 1 million observations, a dozen variables. I recall that in a discussion another one has the following comments: In structural models like ...
0
votes
0answers
27 views

Can't get a Keras model to overfit [duplicate]

(Full disclosure, this is a follow-up to this question, which wasn't completely answered on StackOverflow) The input dataset is a time series of some stock price movement, but it might as well be ...
0
votes
1answer
29 views

Why can having more “model” parameters (weights) in neural networks lead to overfitting?

In http://cs231n.github.io/convolutional-networks/, it states that a "huge number of parameters would quickly lead to overfitting" in neural networks. I don't think I quite understand this. The ...
1
vote
3answers
118 views

In Bishop's textbook, is the example of overfitting exaggerated?

Here, the data $x$ are randomly generated, and $t$ are generated by running $x$ through a function $\sin(2\pi x)$, then Gaussian noise is added. Bishop's text then tries to fit those data using a ...
2
votes
1answer
27 views

Best strategy to maximize the prediction accuracy when p >> n

I am solving the following classification problem: thousands of features, but only 40 samples (i.e. p >> n) classes are balanced it is not possible to get more data the only thing I am interested in ...
0
votes
0answers
20 views

Difference of network between testing and training on the same dataset (No training and testing)

I was training and dense net model on emotion recognition on the sewa dataset. Therefore, at the end I have 2 outputs. One for arousal and the other for valence (These dimensions for emotions). So I ...
0
votes
1answer
24 views

Can overfit happen in spite of validation and what to do with it?

Let's consider a standard situation where we need to find a predictive model. We train all the available model using a training data set. We validate all the trained model using a validation data ...
0
votes
1answer
14 views

How does train-validation-test procedure deals with the sampling error of the accuracy measure?

Let's consider a standard model selection procedure: We have N different untrained models (for example linear regression, neural network, decision tree and so on). We use a data set A to train each ...
0
votes
0answers
34 views

High dimensional regression overfitting

Consider the linear regression model \begin{equation} \boldsymbol{y} = \boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{\epsilon} \end{equation} where we assume $\boldsymbol{X}$ is $n$-by-$p$, with $p &...
0
votes
0answers
27 views

Why do we take `(Bias) ^2` in total error in a model? [duplicate]

I was recently studying some book and few blogs and come to note that : Total error = Bias^2 +Variance + irreducible error Also, I know that these are the errors ...
0
votes
1answer
55 views

When does my unsupervised autoencoder start to overfit?

I am working on anomaly detection using an autoencoder neural network with $1$ hidden layer. This is an unsupervised setting, as I do not have previous examples of anomalies. The input data has ...
1
vote
1answer
35 views

Fitting model on whole dataset, more or less epochs ? (w.r.t validation accuracy) [duplicate]

When tuning my neural networks hyperparameters I use 20% of the data set as validation data. With the holdout set I observe the validation accuracy and validation loss. In my case the model starts ...
1
vote
1answer
195 views

Finding the appropriate polynomial fit in Python

Is there a function or library in Python to automatically compute the best polynomial fit for a set of data points? I am not really interested in the ML use case of generalizing to a set of new data, ...
1
vote
1answer
35 views

With test accuracy being equal, is it better to have lower training accuracy?

Suppose we train two models on a training set, and then test them both on the training set itself, and on a test set. We have some accuracy metric we're using to evaluate them. Both models score ...
1
vote
1answer
23 views

Distinguishing between overfitting and wrong model selection

I built a dozen of different models using caret package for classifying customer purchase habits into 5 categories (catA, catB, catC, catD, none) based on 4 numeric ...
3
votes
1answer
67 views

why too many epochs will cause overfitting?

I am reading the 《deep learning with python》. In chapter 4, about Fighting overfitting, I have two questions. why increasing epochs may cause overfitting? I know ...
0
votes
1answer
49 views

Emotion detection: neural network overfitting on audio files

I am working on an analysis of audio data to understand emotions using the RAVDESS dataset. The input is the Mel-frequency cepstral coefficients (MFCCs) of each audio file, extracted using a Python ...
0
votes
1answer
20 views

Should the validation set have the same ratio in the categories as the whole data?

I'm currently working on a classification problem. The variable Y in 70% of cases is 0 and in 30% of cases is 1. Does my validation set have to have this same proportion? I ask because after using ...
0
votes
0answers
55 views

Cox proportional hazards model, problem with correlated predictors or overfitting?

I have a question concerning a Cox proportional hazard model (in R) on which I would love to get your opinion and feed back! I think that there is a problem, however I would like to be sure and ...
1
vote
1answer
85 views

Why do we use multiple epochs and why does it not lead to over fitting

I have found many answers to the question of what an epoch is, but none to this: Why do we use multiple epochs when training a neural net? How does this not lead to overfitting? From my ...
0
votes
0answers
8 views

Dataset that overfits with increasing test error curve

I want to test some optimization algorithm for DNN to see if it prevents overfitting. I'm looking for a images dataset that overfits with SGD s.t. test 0-1 error curve increases with iterations. If it ...
0
votes
0answers
42 views

How does a covariance matrix over fit if we have too few data points?

I am reading this : - Honey I shrunk the Sample Covariance Matrix The author on page 2 says that : The crux of the method is that those estimated coefficients in the sample covariance matrix that ...
1
vote
0answers
27 views

SVM model overfitting?

I have a multi-class (10 classes) classification problem. I am using one-vs-rest SVM modeling with sklearn.svm.SVC. I want to know whether my model is over-fitting. For train set accuracy is 100% ...
0
votes
1answer
63 views

Is the best model always one with best test score, even though it looks overfit?

I'm making a binary classification model using gradient boosting (lightgbm). I usually use learning curves to check if my model is overfitting. The metric I'm using is sklearn's average precision-...
0
votes
1answer
75 views

Is this a sign of (bad) local minima encountered by this RNN?

On a binary classification task, I can get perfect performance on the training set, but no matter how strongly I regularize the recurrent neural network (dropout 0.99, L2 weight penalties of 0.01) the ...
2
votes
1answer
47 views

Is RJMCMC robust to overfitting?

I have noticed that RJMCMC is often described as robust to overfitting. I am struggeling a bit with the intuition for this. Why doesn't the Reversible jump Markov Chain Monte Carlo (RJMCMC) always ...
1
vote
1answer
78 views

How to test for overfitting in a TAR model in R?

I want to fit a threshold autoregressive model, and I'm using the tar package in R. For ARIMA models, I could check if a model was overfit by looking at the values of standard errors as compared to ...